For that reason, involving people in the decision about who provides feedback and spending time helping them make the best choices will have a tremendous pay-off later on. As one manager said after he looked at his feedback report, “If I had known how useful this was going to be, I would have been much more careful about choosing people to complete the questionnaires.”

Suggest that they narrow the pool of raters by selecting people

While many organizations allow feedback recipients to choose their own raters, some prefer to pre-select respondents to ensure an unbiased and representative distribution. In these organizations they are skeptical about the possibility that people might be able to manipulate their profiles by selecting raters they know are favorably disposed towards them. They wonder to what extent the ratings people receive are consistent with their supervisors’ view of them.

Pre-selection of respondents can, however, make participants feel that they have less control over the process, which decreases their commitment and the likelihood that they will accept their feedback and feel motivated to change. While allowing people to choose their own respondents can sometimes mean that participants will distribute questionnaires only to their friends, if the feedback is confidential, even friends will usually provide honest responses that indicate where improvement is needed.

As it turns out, there is evidence that supports this point of view. During the 1980s, a consulting firm conducted a series of validity studies for Disney that attempted to determine the fairness, accuracy, rater bias, and popularity of the 360-degree feedback process that had been used at such Disney properties as Disneyland, Disney World, Corporate, Disney University, EPCOT, WED, and Disney Studios.

At the time that the viability and fairness studies were being conducted at EPCOT, the general manager there was a Ph.D. statistician. He was unimpressed by others’ perception of the fairness of the process and argued that its validity had to be flawed by cast members’ involvement in the selection of their own evaluators.

Based on this concern, the consulting firm conducted another multi-source assessment project at EPCOT, specifically designed to test the general manager’s hypothesis. A total of twenty-two cast members selected two different evaluation teams, one composed of their friends and the other of “grumpies.” The idea was to see how much the two sets of raters varied in their evaluations.

To the surprise of the skeptics, the variations in the two sets of profiles that emerged were remarkably small. Only two profiles showed more than a 7 percent difference between one evaluation team and the other. The clear indication was that the 360 process was sufficiently robust to allow feedback recipients to be involved in respondent selection without skewing the results.²

“colleagues/peers,” and 55 percent checked “boss.” Where “enhanced” 360-degree feedback is the chosen approach, data are gathered not only from these sources but also from the person’s family members and friends, psychological profiles, early work history, and childhood experiences.³

Customer feedback is widely regarded as extremely valuable and powerful, since the customer’s perceptions and expectations are of key importance for many organizations. However, only 33 percent of respondents in our survey indicated that “external customers and partners” were included as raters in their 360 feedback process. This is not too surprising given that eliciting customer feedback can lead to a number of administrative problems. For example, since the behavior an external customer has a chance to observe may be different and more limited than that seen by the person’s immediate co-workers, different questions are needed, and the resulting data have to be compiled and presented separately. There is also a risk that the customer will perceive requests for feedback as onerous, time-consuming, and disruptive to his or her business, especially if the request is not made tactfully, or the benefits to the customer are not made clear. And involving the customer requires the feedback recipient to be diligent about following up on next steps and relevant joint actions.⁴

For one senior manager, we were asked to design a process to include bosses, direct reports, peers, and customers in his data gathering. We used a similar tool for the bosses, direct reports, and peers, but totally customized the data gathering format for customers. The overall purpose was to identify ways for the manager to improve his strategic impact on the business, so we were able to focus on his strategy development and team leadership internally and focus on how he communicated and executed that strategy externally. In addition to clarifying what he was doing effectively and what required change inside the business, he was able to work in partnership with his customers to improve relationships and establish stronger ties between their respective organizations. One customer described the process as the best form of customer engagement he had ever experienced.

Because raters will be providing feedback on what they have observed about a person’s behavior, their data will be more accurate and appropriate if they have worked with that person for a reasonably long time. Therefore, it is a good idea to require that each rater have worked with the recipient for at least four to five months.