Upward evaluation at the Arizona Health Sciences Library*

Abstract

At the director or supervisors' supervisor level, upward evaluation can provide valuable input regarding the supervisory skills of the supervisors they oversee. Because of its potential benefits, the AHSL decided to develop and implement a process that incorporated evaluation of supervisors by subordinates into the annual performance review process.

Upward evaluation is a process that involves the evaluation of supervisors by those they supervise. Although common in industry [1, 2], it is fairly rare in libraries and even more rare, though not unheard of, in medical libraries [3]. A recent informal, unscientific poll of academic health sciences library directors conducted by the Arizona Health Sciences Library (AHSL) director revealed that only three of twenty-six respondents participated in upward evaluation of managers. Three others had a process for evaluation of the director only.

During the summer of 2007, the AHSL explored the many potential benefits of upward evaluation and concluded that benefits could accrue at three different levels. According to the literature, at an employee level, upward evaluation gives employees a voice in the organization of which they are so integral a part, can lead to better staff morale and improved commitment, and can reduce conflict [1, 4]. At the supervisor or middle management level, it can serve as a powerful tool for professional development. At the director or supervisors' supervisor level, upward evaluation can provide valuable input regarding the supervisory skills of the supervisors they oversee. Because of its potential benefits, the AHSL decided to develop and implement a process that incorporated evaluation of supervisors by subordinates into the annual performance review process.

BACKGROUND

The University of Arizona requires annual performance appraisals, in other words, supervisors evaluate the individuals whom they supervise. In addition, the standard evaluation forms ask evaluators of supervisors to score and comment on leadership, staff relations, and performance appraisal. However, it was sometimes difficult to evaluate staff members in areas that could not be observed directly or consistently. At the same time, some library staff members felt that they could make a valuable contribution to the evaluation process and improve the performance of their supervisors if there were a means for them to participate. Consequently, the AHSL designed a procedure that made evaluation a two-way process. Although sample guidelines from the University of Nebraska [5] and forms from both the University of Houston and the University of Arizona Main Library (internal documents) were found, none could easily be incorporated into the AHSL's existing process. The university's human resources (HR) department was supportive and suggested using a modified version of the "360 Review" commonly used to evaluate university administrators.

Having gained the support of HR, the director next gauged the level of general interest among the fortytwo library staff at an all staff meeting. The response was overwhelmingly positive. With input from the library leadership group, he then formed a committee of seven with representation from all library departments consisting of three library faculty members (one supervisor and two non-supervisors) and four classified staff members (one supervisor and three non-supervisors).

The committee began by developing an evaluation form. Rubin wrote that the ideal instrument for upward evaluation should possess at least five characteristics: the staff should be significantly involved in its creation, it should be easy to both fill out and interpret, it should take little time to complete, it should encompass all the major points needed to evaluate a supervisor, and it should be designed so only the supervisor's supervisor would have access to it [6J. When the committee reviewed the template that supervisors' supervisors were already using to evaluate supervisory skills of middle managers [7], they found that all of the above criteria were met. Additionally, the committee reasoned that by using the exact same questions supervisors' supervisors had to answer, the form would help meet one of the three primary objectives of upwards evaluation: to provide the supervisors' supervisor unique and direct insight into skills and behaviors that the supervisors' supervisor previously obtained indirectly or inferred from the performance of that supervisor's department [8]. After minor adjustments were made so that both faculty and staff forms were identical, the revised form requested employee feedback on the following qualities in their supervisors:

* leadership

* scheduling

* judgment

* staff relations

* staff development

* performance appraisal

* cost effectiveness

* commitment to diversity

* communication

Given the small size of the library, the committee decided a purely qualitative approach would provide the most valuable feedback. Even in situations where staff size makes the gathering and statistical analyses of quantitative data meaningful, authors have emphasized the value of qualitative data [2, 8]. AHSL's questions about the attributes outlined above were accordingly very open ended. Employees were asked to provide comments and examples, unless they truly had no basis on which to judge, in which case they could check "Not applicable /Unable to observe."

Worries about confidentiality emerged as the central concern in the AHSL committee discussions. This is echoed repeatedly in the literature [2, 7-9]. Confidentiality can be especially hard to ensure where the format is qualitative rather than quantitative, because, as Turrentine et al. found, "employees reasonably believed that their words could identify them in ways that a numerical response could not" [2].

Not only might employees worry about possible consequences such as supervisor retribution, but supervisors, too, worry about unbridled employee responses and about the impact of such responses on their evaluations and therefore on their employment status. It should be noted that the literature stresses that upward evaluation is a tool to be used exclusively for professional development and performance improvement, and not for personnel decisions [2, 4, 9, 1O].

Other considerations debated by the AHSL Upward Evaluation Committee included whether or not filling out the supervisor evaluation form should be mandatory; whether or not all employees should participate - and if not, who should and should not - and why; and finally, should the forms be signed? The committee did ultimately decide to make the forms mandatory and to have all employees participate. Individuals could also additionally provide feedback about someone who was not formally a supervisor but who acted as a project leader (a dotted-line reporting relationship). The committee's thinking was that the library is small enough that, for the procedure to engender truly meaningful information, full participation would be required.

The committee also decided to have employees sign their forms. While this might seem counterintuitive in the light of the overarching concern about confidentiality, the committee's thinking was similar to that of the library director at Virginia Polytechnic Institute and State University (Virginia Tech) described by Linkins, who wanted the upward evaluation forms at his institution signed for two reasons: "He believed that responsible people should be willing to stand behind their words. More importantly, from his point of view, he wanted the flexibility of interviewing individuals whose evaluation of their department head indicated problems needing further investigation" [9]. Similar to Virginia Tech, where signed evaluations were seen exclusively by the director and associate director [9], and the University of Kentucky, where signatures were placed on a detachable card that was discarded after an initial review by the director [4], AHSL employees were asked to sign their evaluations. Once their supervisors' supervisors incorporated the feedback into their (downward) evaluation of the supervisors, the forms were shredded and any electronic copies were deleted. Furthermore, the supervisors' supervisors were asked not to use direct employee quotes in their evaluations.

These decisions and the discussions that led to them were presented at a second all staff meeting, so that everyone could have the opportunity to offer input and be involved in the final decision-making process. This open dialogue increased staff awareness as to what was taking place, gave them an opportunity to voice their concerns, educated them as to the process that would be taking place, and promoted buy-in for the concept.

THE SURVEYS

After upward evaluation was in place for two years, three web-based surveys were conducted to solicit feedback on the process: One survey was directed at both current and former employees who evaluated their supervisors (n=34); a second solicited feedback from supervisors (n=8); and a third went to the evaluators of supervisors (n=3).

Survey #1 was emailed to current staff, and letters were sent to former employees asking them to participate and providing them with the uniform resource locator (URL) for the survey. Fifteen individuals responded (44%). Table 1 summarizes their responses.

Despite a mixed response at the employee level, 80% of respondents wanted to see the process continue (Table 1). Of the 15 respondents, 1 had consistently negative comments, ranging from "Confidentiality didn't seem to be honored" to "There was even less communication with me. It was put in my ball court that it was my fault and there was no problem with the supervisor." However, this respondent also wanted to see the process continue. Of the 2 respondents who did not want to see the process continue, 1 was completely dissatisfied with the whole procedure but left no comments and the other was completely satisfied and felt that his or her "supervisor's behavior is fine." Two individuals commented that while they had noticed changes in their supervisors' behavior, they could not definitively attribute that change to the upward evaluation process because other things had also changed in the library. Another individual reflected that even though she or he felt that confidentiality was maintained, feedback might still have some unintended consequences, preventing that employee from being as forthright as she or he would have liked.

The second survey asked supervisors if their supervisory skills had changed after the upward evaluation was implemented. One of the 4 respondents noted that comments made about his or her supervisory skills during the evaluation process were more specific and more positive than previously. None of these supervisors, however, had changed the way they supervised based on the upward evaluation. Supervisors were also asked if they could trace any of the comments to any one employee, because confidentiality was such a central issue in the development of this procedure. One respondent felt that he or she was able to identify comments as coming from a certain individual. When asked if they wanted to see the procedure continue, 75% of the respondents were in favor of continuing upward evaluation with 1 person saying: "I believe it holds you accountable to those you supervise."

The third survey was developed for supervisors of supervisors. Three people in the library fell into this category, two of whom responded to the survey. While both felt that it was easy to maintain the confidentiality of evaluators, they were split on whether the staff evaluations were helpful. One supervisors' supervisor specifically stated, this process "brought new issues to my attention. Confirmed my opinions in some situations. Given me additional insight in many cases." One supervisor definitely wanted the process to continue, while one was indifferent.

CONCLUSIONS

The overall positive experience at AHSL seems to reflect one of the primary benefits of upward evaluation: to give employees a voice and an increased sense of ownership in the organization of which they are a part. As Starratt and Tollman so aptly state, "We all like to have our opinions solicited, and can feel much better about even the worst situation if we feel we have 'done something' about it, and that our opinions matter" [8].

At the supervisor/middle management level, it was hoped that upward evaluation would provide an instrument for professional development. As none of the supervisors at AHSL indicated that they were supervising any differently, this particular benefit of upward evaluation remains hypothetical. Perhaps further refinements of the process at this level, such as focused coaching by the supervisors' supervisor, would help realize some of the great potential for development in this area.

Finally, at the director or supervisors' supervisor level, upward evaluation offers an avenue by which insight into the supervisory skills of middle management supervisors may be gained. Although the AHSL sample of two out of a possible three respondents was split right down the middle, the benefit to one was unequivocal: Upward evaluation reinforced some opinions by bringing new issues to his or her attention and added insight regarding other issues. A positive result for traditional downward evaluation was also reflected in a middle management supervisor's comment that "Statements on my supervisory skills were more specific and more positive."

The number of employees at the AHSL is relatively small, and the number of survey respondents smaller still. Although the resulting sample size makes it difficult to reach definitive conclusions regarding upward evaluation, there seem to be few downsides and many potential benefits. Rubin quotes Roñan (1970) as asserting that "the most important factor in employee satisfaction is the immediate supervisor" [U]. Upward evaluation provides a continuing and evolving opportunity to improve the relationship between employees and their supervisors and as such to improve the morale and performance of all.

Footnote

* Based on a presentation at the 2010 Joint Meeting of the Medical Library Group of Southern California and Arizona (MLGSCA) and the Northern California and Nevada Medical Library Group (NCNMLG); Glendale, AZ; January 29, 2010.

References

REFERENCES

1. Bernardin HJ. Subordinate appraisal: a valuable source of information about managers. Hum Resour Manage. 1986;25(3):421-39. DOI: 10.1002/hrm.3930250307.

2. Turrentine CG, Lener EF, Young ML. A qualitative approach to upward evaluation of leadership performance: pros and cons. J Acad Libr. 2004 Jul;30(4):304-13. DOI: 10.1016/^31^.2004.04.004.

3. Martin] A. Staff evaluation of supervisors. Spec Libr. 1979 Jan;70:26-9.

4. Perkins GH. Enhancement of organizational structure through upward evaluation. Libr Adm Manage. 1992 Fall;6(4):198-202.

5. University of Nebraska. University of Nebraska policies: performance evaluations [Internet]. Lincoln, NE: The University; 1 Aug 2001 [cited 6 May 2010]. <http://bf.unl.edu/hrpolicy/PerformanceEvaluations.shtml>.

6. Rubin RE. The development of a performance evaluation instrument for upward evaluation of supervisors by subordinates. Libr Inf Sci Res. 1994 Fall;16(4):315-28. DOI: 10.1016/0740-8188(94)90055-8.

7. University of Arizona. Performance evaluation forms [Internet]. Tucson, AZ: The University; Dec 2006 [cited 6 May 201O]. <http:// www.hr.arizona.edu /forms#perform>.

8. Starratt JA, Tollman TA. Upward evaluation of library middle managers. Ref Serv Rev. 1986 Mar;14(l):87-90. DOI: 10.1108/eb048931.

9. Linkins GC. Department head evaluations: the Virginia Tech Library experience. J Libr Admin. 1985 Jan;5(4):53-60. DOI: 10.1300/J111V05N04_06.

10. Perkins GH. The value of upward evaluation in libraries: part II. Libr Adm Manage. 1995 Summer;9(3): 166-75.

11. Rubin RE. Upward appraisal: what do subordinates consider important in evaluating their supervisors? Libr Inf Sci Res. 1995 Spring;! 7(2):151-61. DOI: 10.1016/07408188(95)90019-5.