Systematic performance appraisals are needed for ensuring the competency of professional nurses. Peer review involves the use of established standards of practice for the evaluation of a nurse's practice by an individual of the same rank or standing in the organization.1 The use of peer review is associated with increased autonomy, accountability, leadership development, and a better work environment.2-4 However, it can be challenging to implement an effective peer-review process. Locating a valid and reliable instrument is difficult, and even if one is found, there is often insufficient psychometric data to assess its utility.5,6 Moreover, peer appraisals may also result in a sense of injustice and reduced motivation.7,8 Poorly conducted performance appraisals may cause employees to become dissatisfied, less committed to the organization, and more likely to consider leaving their jobs.9
The nurses in our community hospital were not happy with their peer-review process. They complained that evaluations were inconsistent and sometimes unfair or inaccurate. The performance appraisal instrument was described as being too long and lacking in clarity. There were also inconsistencies among units in both the interpretation of standards and the procedures used for reviews. The chief nursing officer found a solution to these problems in a project that involved empowering the hospital's research committee, a group of clinical staff nurses, to define nursing job standards and lead the process of peer review. The project took a year to accomplish, but it completely transformed performance appraisals.
Preparation and Planning
The committee began by studying and discussing multiple information resources including published standards of practice, codes of ethics, evaluation instruments from other hospitals, and articles about peer review, instrument development, and shared governance. The research coordinator provided instruction about concepts related to performance appraisal, instrument development, and measurement issues. During this time, 2 staff nurses emerged as leaders, and they took responsibility for overseeing and coordinating the work.
The group decided to adopt the Synergy Model as an organizing framework for the instrument. This model is used as the theoretical basis for nursing at the hospital, and everyone is familiar with its concepts and terminology. The model defines 8 essential competencies for nursing practice10 and the committee wanted to make sure that the new instrument would address each of these competencies. A formal protocol with an action plan was developed, and approval was obtained from the hospital's institutional review board because the plan included psychometric testing through mock performance reviews.
Developing and Testing an Instrument
The committee began by making basic decisions about the purpose and format of the performance appraisal. They were firm in their belief that the focus of peer review should be to provide feedback to support professional development. They wanted to avoid the use of a numerical scale that might be perceived as a "grade," so they decided to use 3-point scale of "does not meet," "meets," or "exceeds" standards. They generated working definitions of the 4 levels of the clinical ladder and agreed that items on the instrument should be specific, concrete, and descriptive of visible behaviors whenever possible.
The 14 members of the committee divided into 4 groups and the small groups worked independently on developing items for different sections of the instrument. When the entire committee came together again to review and discuss their work, it was discovered that nurses working in diverse settings around the hospital had quite different views about nursing. It became clear that defining specific standards that would be interpreted similarly in diverse clinical units was going to be challenging. During several lively meetings, potential items for the instrument were discussed and revised until a final version emerged that represented the consensus of the group. The staff nurse leaders facilitated the discussions at the meetings and worked with the research coordinator between meetings to incorporate the group's comments into revisions.
The final instrument included behavioral descriptors for all of the levels of the clinical ladder for each of the 8 competencies. Nurses were to receive 1 rating for each competency.
This first complete draft was shared with the chief nursing officer, vice president for human resources, and director of education. These 3 people attended a committee meeting to discuss the proposed instrument, and a few minor modifications were made in response to their suggestions.
Instrument testing began with the measurement of a content validity index of 0.93 using a panel of 9 raters and the process recommended by Lynn.11 A few minor revisions were made to items in response to comments written by the reviewers. Interrater reliability was tested on 2 large nursing units. Two staff nurses were recruited from each unit, and they independently performed mock reviews on 25 of their peers, resulting in a total of 50 paired measurements. Evaluations were coded so that the identities of the people being reviewed would not be known by the research committee. The resulting interrater reliability was 0.85 for the 8-item instrument. Test-retest reliability was examined by asking staff nurses from different units to use the instrument to evaluate their peers on 2 occasions, 2 weeks apart. One hundred five repeated evaluations were completed, and test-retest reliability was measured at 0.94.
Pilot Testing
The instrument was pilot tested in 2 units. A group of 6 reviewers was identified in each unit, and these reviewers participated in a 4-hour class conducted by the director of education. Concepts addressed in the class included the purpose and expected processes, benefits, common concerns, and challenges encountered in performance appraisals. Everyone was given the opportunity to examine the new instrument, ask questions about it, and practice with it in hypothetical evaluations.
Peer evaluations were conducted in meetings held away from the clinical units. The committee decided that directors should not be present during these meetings so that staff nurses could feel confident that the appraisals were from their peers rather than their supervisor. Members of the research committee served as facilitators and recorders, and the resulting written evaluations were given to the directors, who then met with each nurse individually to share the results.
A brief form with open-ended questions was developed to elicit feedback from the 12 peer reviewers, and 10 anonymous forms were returned. The 2 directors provided verbal feedback. Comments about the instrument and the standards it defined were 100% postive. The instrument was described as clear and easy to use. However, the directors felt that the narrative feedback lacked sufficient detail to provide the basis for a meaningful evaluation conference that would support professional development. This was compounded by the fact that directors were not present to hear the discussions of the reviewers.
In response to the findings from pilot testing, the committee made changes in both the instrument and the process. They decided that it was necessary for directors to be present during evaluation discussions but recommended that they should act only as facilitators and recorders. The instrument was modified to add structured questions for written feedback and to require specific examples of observed behaviors to support ratings. The peer-reviewer preparation class was expanded to add content related to writing narrative feedback and provide more practice with doing this.
Outcomes
The appraisal instrument (see Document, Supplemental Digital Content 1, http://links.lww.com/JONA/A47) serves as the job description for staff nurses, and it has been used throughout the hospital for peer-review appraisals for 450 nurses. There is anecdotal evidence that the reviews have stimulated professional development. For example, 1 nurse received feedback that her peers did not believe she was meeting many of the standards associated with her clinician IV status. Within a few weeks, she initiated a process improvement project, gathered research evidence to update a protocol, and became more active in shared governance. One interesting occurrence was that the unit directors perceived that the standards and job expectations had been significantly elevated from the previous requirements, and they thought there would be a negative reaction to this. However, there has not been a single complaint about the higher standards. The professional practice committee responded to the new standards by streamlining their guidelines and procedures for promotion on the clinical ladder, and this has stimulated an increase in the number of nurses seeking promotion.
This project provided a unique opportunity for staff nurses from diverse specialties to work together to define nursing performance standards for our hospital. The resulting instrument reflects the values, expertise, and culture of our clinical staff nurses, and their work has transformed the peer-review process.
References