Lack of Agreement on Performance Criteria. Anyone who has participated in a performance appraisal as either a boss or a direct report will not be surprised to hear that the two parties do not always agree on what it takes to perform the job effectively. Our own experience supports this conclusion. Part of the data we collect with our 360-degree instrument, “Compass: The Managerial Practices Survey,” consists of importance ratings for the fourteen practices related to effective performance. The boss rates which practices he or she feels are essential for the direct report’s effective performance of the job, and the direct report does the same. Their perspectives rarely coincide at every point.

If bosses and employees cannot come to an agreement on what is required for good performance, it is hardly surprising that they come to different conclusions about the employee’s effectiveness in meeting performance goals. In addition, if the appraisal focuses only on the ability to accomplish goals or outcomes, without reviewing the behaviors required for successful performance, it does little to close this gap in perception or help the employee understand what changes are required for improvement.

The Inability to Handle Lots of Information. The thought process behind an effective appraisal is very complex. Experts break down the intricate process for collecting and processing information used in the appraisal process into observing behaviors, translating these observations into a cognitive representation, storing the representation in memory, retrieving the stored information, integrating the stored information with other data, and finally, assessing and evaluating the data.⁹ One expert has concluded that the complexity of the process and the limitations on human information-processing capabilities often cause bosses and employees to simplify the task and work from overall impressions rather than specific details.¹⁰ In other words, because each of us uses his or her own framework to process and remember information, we attach more weight to information that fits our model of the world and discount information that is inconsistent with that model. These different approaches to remembering and processing large amounts of information contribute to a potential lack of agreement between the employee and boss on the final evaluation.

Concern About Self-Image. For some people, appraisals can be an opportunity to reaffirm their value to the organization and celebrate their growth and accomplishments over the previous year. For others, appraisals can be extremely threatening—contentious, demoralizing, and anxiety-producing encounters that bring all the employee’s defense mechanisms into play. Defense mechanisms protect our self-concept; as a result, we can generate a subjective and distorted view of our contributions or results. When people devote energy to denying or explaining away performance problems in order to preserve a positive self-image, agreement on the evaluation will be difficult.¹¹

Gain Agreement on Expectations. The addition of a 360-degree process can help minimize the differences in perspective by ensuring that the boss and direct report work from the same model of effective behaviors and outcomes. Comparing boss and direct report ratings of the importance of critical success factors and using these data as a springboard for discussing what it takes to succeed in a position can bring the two perspectives significantly closer together.

Use a Broader Range of Information. The use of 360-degree feedback provides a more objective measure of a person’s performance. Incorporating the perspective of multiple sources provides a broader view of the employee’s performance and helps minimize biases that result from not only limited views of behavior but limited information-processing capabilities. In addition, because the results of a 360-degree process are usually documented in a report, there is less dependence on the boss’s and employee’s memory and retrieval mechanisms.

Facilitate Open Discussion. Using 360-degree feedback can play an important part in keeping the employee open to the appraisal discussion. Studies have shown that people are more likely to modify their self-perceptions in the face of multi-source feedback.¹² Multiple views of a person’s behavior also increase the likelihood that important elements of performance will not be overlooked or minimized, and a well-run appraisal meeting in which the boss provides balanced feedback in a constructive manner reduces the employee’s defensiveness during the appraisal process.

Before we discuss how to make 360-degree feedback work in the appraisal system, we must offer two caveats. First, the information you collect and how you collect that information should be different for use in appraisals than for use in development. Second, the addition of 360-degree feedback will not fix a traditional appraisal system that is not currently working. Although 360-degree feedback can be used to address some of the issues related to fairness and accuracy, it will not ensure that people have open and honest discussions about the data and identify appropriate next steps.

The following guidelines and suggestions can help ensure that the use of multi-source feedback actually improves your appraisal process.

Involve People. Having recipients participate in various aspects of the appraisal process will strengthen their commitment to the overall process and increase their commitment to the decisions that get made. Have recipients jointly clarify performance expectations and set goals, ask them who should serve as raters (sometimes people outside the organization will be named), and give them an opportunity to review and interpret the results.

Ensure That Relevant Data Are Being Collected. Focus on behaviors and outcomes that raters are capable of observing and are competent to evaluate. This increases people’s confidence in the quality of the data and their usefulness as part of the overall assessment. Such confidence is also increased by allowing people to participate in the selection of the raters who will evaluate them.

Ensure That No One Rater Can Affect the Outcome. Be sure to use a large enough sample or eliminate the highest and lowest scores to ensure that no one rater can skew the data.

Train Raters in What and How to Observe. The better people understand what to look for and how to record critical incidents that can be used as examples to support their ratings, the better the quality of the information that will be collected. Trained raters provide clearer, more detailed information that contributes to understanding the data and to developing effective plans for improvement.

Have Raters Support Their Evaluations. This suggestion is especially appropriate for high-performance or self-managed teams. We believe that, whenever possible, raters should discuss their observations and evaluations with other raters. Because of the complexity of the information processing required to arrive at an accurate assessment, the appraisal process benefits from discussion—what others say triggers thoughts and prevents selective memory. We also believe that raters should be required to provide a rationale for their rating to other raters. Such discussion not only enlightens others, but it also highlights biases and prevents people from using the process to act on personal grudges.

Move Slowly and Start Small. We strongly recommend that any organization considering using 360-degree feedback in the appraisal process begin by using it for development only and gradually make it part of appraisal discussions with a pilot group. Even then, the focus should be on the goal-setting portion of the appraisal. People need to be comfortable with the idea of multisource feedback as a development tool. They frequently benefit from experiencing the feedback process and gaining an appreciation for its objectivity. Once this hurdle has been overcome, there will be less resistance to the use of 360-degree feedback in broader appraisal discussions that include the evaluation itself.

If you answered “no” to any of the above questions, your organization may not yet be ready for this step.