Choosing the Method That Will Work Best for You
Each method has its strengths and weaknesses. Although both are highly effective, questionnaires tend to be more widely used because they are easier to administer and less costly. They are also easier to score and have greater (or at least better established) reliability and validity. As you think about the method, or combination of methods, that is most appropriate for you, you will need to keep in mind the number of people who will receive feedback, the organizational level with which you are working, the type of data you will be collecting, and the resources you have available—budget, time, personnel, and expertise.
The Number of People Who Will Receive Feedback
Let us say you are working with an individual manager, Andrea Johnson, on her personal and professional development and have decided, with her agreement, that 360-degree feedback would be a useful way to gain insight into her behavior. Ms. Johnson is the leader of a business team consisting of five direct reports. Along with seven other colleagues, she is a member of her boss’s business team; she also works with dozens of colleagues in other functions and business units across the company. These activities give her many possible sources for feedback. At a minimum, you want to gather perceptions of her behavior and effectiveness from about ten people—four direct reports, five colleagues, and her boss (or some similar combination).
For each manager slated to receive feedback, you will have to collect and analyze data from ten people, not including the manager. If you are working with a group of ten managers, you will need to collect data from at least one hundred people. As you can see, the magnitude of the effort can be daunting. So what is the best approach—interviews or questionnaires?
When you are working with large populations, questionnaires are a far more efficient method for collecting data, as many are web-based and only need to be distributed with instructions for how to complete them and when and where to submit them. You can easily involve greater numbers of respondents, although we find that eight to ten people are usually sufficient. The time required to collect the data is significantly less than with the interview method, and the data analysis is faster, because it is based on quantifiable items and done with a computer-based scoring program.
Level in the Organization
For individuals or small groups of senior executives and high-potential middle managers, you may want to invest the time and money required for the interview method. The flexibility of the interview approach enables you to get at the unique aspects of the senior executive’s role and the demands of the job and provides rich anecdotal data that help people better understand how they are seen by others. The presentation of the findings, which usually takes place in a one-on-one meeting, allows for a more in-depth discussion of strengths, weaknesses, and next steps. Such a setting also provides an opportunity to explore the need for change and examine highly focused strategies for change most suited to people in senior positions. For example, it might enable an executive to go beyond simply identifying networking as an area in need of development to discussing why better networking is needed as he manages the launch of a new business venture.
MetLife implements something they call “Live 360” for development purposes. The target participants are in the most senior roles and the data collection is done solely by interview. As Deb Capolarello, senior vice president human resources, people practices, and chief talent officer, says, “At this stage of their careers, these executives are already performing at high levels. This approach allows us to probe and get at the nuances that are critical for success in the executive ranks. To get to real development needs at that level, you need the insights that go beyond quantitative responses on a questionnaire.” In some cases, the 360 will be combined with an executive assessment to provide for a more in-depth view. Executives understand the purpose, approach, and expected outcomes before the process is set in motion. Once the data are collected, the executive reviews the results with the assessor. The assessor then meets with the executive’s manager to describe the findings. The third step in the process is the meeting between the executive and his or her manager to discuss the results and determine the focus for development. The executive is expected to come to this meeting with a draft development plan and the outcome of the meeting is agreement on priorities. The executive has the option of using an external development expert as a resource and all results from the process are shared with human resources for inclusion in their talent management profile.
Perhaps the most impressive step in this process is that MetLife’s chairman, president, & CEO, C. Robert Henrikson, meets with every executive who participates in this process. In addition to the value of the one-on-one meetings for the executive, Henrikson’s involvement provides evidence of the importance of development to the organization.
The Type of Data to Be Collected
Some types of data are more easily collected by questionnaire, while others are best collected through interviews. If you are primarily interested in gathering information on people’s skills and knowledge—their current abilities—a questionnaire may be the most efficient way to go. The multiple-choice construction of a questionnaire works very well when collecting people’s perceptions of how frequently or how well specific, observable behaviors are being used. While this type of information can also be collected through interviews, and the data may be richer because of the interviewer’s ability to probe for more detail, additional information may not be worth the time and resources you will have to devote to get it.
If the focus of your data collection is on recipients’ potential or their style or personality characteristics, multiple-choice questionnaires may not provide enough information. To get an accurate picture of their ability to learn and to apply what they have learned or to see how their patterns of behavior play out, interviews may be necessary and worth the time and resources they require. Unlike questionnaires, interviews allow you to ask follow-up questions to get at the nuances of a person’s style and its impact on others. It is harder to anticipate the second- or third-level questions you would need to ask to obtain this type of information when developing a questionnaire.
The Resources You Have Available
In deciding which approach to use, you will first need to consider the total time available to complete the project and the amount of time you can allocate per feedback recipient. Obviously, the more recipients you have, the more time you will require to collect, process, and present the findings. Most questionnaires can be distributed, returned, and processed (with reports prepared) within two to three weeks. Presenting the data will take about five working days for every one hundred people (five one-day sessions for twenty people each). If you are using interviews, you will require about five working days per recipient (a half-day to schedule the interviews, two days to conduct the interviews, two days to analyze the data and prepare the report, and at least two or three hours to review the findings and prepare a development plan).
The number of people you can dedicate to the project and the degree of expertise they possess will be another important consideration. As we have indicated, the interview method, although highly effective, is labor-intensive. It also requires a certain level of expertise to conduct an effective interview and make sense of the data that have been collected. Furthermore, for the interviewing and presentation process to be truly effective, the interviewer must have the complete trust of the recipient and the people from whom the data will be collected. It can be expensive to use outside resources for this type of work, either to obtain expertise, to augment internal staff, or to assure recipients and respondents that the data will be treated confidentially.
The questions of time and resources take us to the important consideration of budget. As mentioned earlier, questionnaires tend to be less expensive—the same budget dollars do not go as far using the interview method. With larger groups, volume discounts based on the number of questionnaires can drive the per-person cost even lower.
Apart from budget issues, resource availability is the primary reason people select the questionnaire method—it does not require a lot of skilled personnel to administer or prepare reports of the findings.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.