Manuel LondonJames W. SmitherThomas Diamante
In this chapter, we examine best practices in leadership assessment, with a focus on chief executive officers (CEOs) and senior executives. These executives include heads of multinational corporations, business units, or large divisions in business, government, or not-for-profit organizations; entrepreneurs who started and grew their own businesses; and executives of large or medium-sized organizations who were promoted from within or hired from outside. Our goal is to understand methods for assessing executives’ performance and potential to perform well at a more senior level of an organization.
Executive turnover poses major challenges for corporations. During the first 10 months of 2004, 516 major companies changed CEOs.[1] In the airline industry alone, all seven of the major airlines in the United States changed CEOs since September 11, 2001.[2] Surveys of management performance often find that as many as half or more leaders are not performing up to par in critical areas.[3]
As a prelude, we emphasize that leadership assessment entails measuring process and outcomes. That is, measurements should capture not only the results of a leader’s behavior and decisions but also how the leader accomplished those outcomes. Also, leadership assessment itself is a process. It can be used to track performance improvement and ongoing development. It is part of an ongoing, cyclical process of organizational needs assessment, goal setting, development, appraisal, feedback, further development, and tracking change. Responsibility for assessment is led or directed by top executives, guided by human resource executives, and administered by managers and leaders throughout the organization.
To organize our review of best practices in leadership assessment, we follow a framework that addresses (1) the purpose of the assessment (for instance, whether to measure past performance, determine readiness for promotion, predict performance for selection or placement, and/or use the data for development); (2) what is assessed (competencies for today and the future); (3) challenges relative to context (pressures faced by the executive; degree of current or anticipated change in responsibilities or expectations); (4) assessment methods that focus on what was accomplished (outcomes) and how (process); and (5) assessing assessments—determining their accuracy, validity, and financial value so that assessment results are not misused and organizations have reasonable expectations of their ability to predict leadership behavior. Figure 2.1 outlines the topics we cover.
There are three basic purposes for leadership assessment: (1) prediction, (2) performance review, and (3) development. Regarding prediction, assessment may be used to select executives from within or outside the organization, place executives in key positions, determine whom to promote, or determine whom to lay off. Outsiders, such as venture capitalists, may assess members of the executive team of a company to help potential investors determine whether the firm is a sound investment. Assessment may also be used to make administrative decisions, such as how much bonus pay a person deserves. Assessment may evaluate readiness to advance, such as determining the individual’s ability to function effectively in different situations and handle challenges. As such, assessments may evaluate potential and leadership emergence.
Regarding performance review, the dimensions of performance that are used for assessment send a message to the individuals assessed about what is important in the organization—what gets measured and rewarded. Goal setting and assessments can be a way to align organizational, team, and individual goals in the form of competency models and balanced scorecards.
Another use of assessment as appraisal is to enhance performance. That is, the goal is to achieve organizational excellence through performance management, which includes appraisal and feedback. In addition, assessment is a vehicle for holding executives accountable to different stakeholders (executives in the parent company, the company’s board and stockholders), each of whom may have different criteria and expectations for performance.
Regarding development, assessment provides executives with feedback that points out strengths and areas for improvement. Executives often have a blind spot when it comes to recognizing their own performance. Higher-level executives tend to have an inflated view of their emotional intelligence and less congruence with how coworkers view them.[4] While high performers tend to match the perceptions and ratings of others, leaders often do not receive the feedback they need to understand how they are perceived by others. Upward feedback from subordinates is limited, since employees may see risks to their careers if they share feedback about problematic behavior of their superiors. At the executive level, many of one’s peers are in reality competitors, and therefore there is little incentive to share feedback that might improve a competitor’s effectiveness. So executives are likely to inflate their self-evaluations, and raters provide middle-of-the-road ratings. Research and practice needs to address how to incorporate performance feedback as a routine part of daily practice, rather than a once-a-year main event with potentially threatening consequences.
There are three basic steps to developing an assessment:
Gain a solid understanding of the job requirements (the actual position) and the needs of the organization. This can be accomplished through interviews with key staff, including those who are in similar positions, or using any of the well-established approaches to job analysis.[5]
Relate these requirements to personal characteristics (elements of character, values, motivation, or personality) or behaviors (for example, strategic vision, charisma, stress resilience, coalition building, communication, people development).
Develop methods for measuring these qualities. Developing a competency model entails identifying, defining, and measuring characteristics that are associated with successful job performance.[6] Competency models have the advantage of clarifying what skills and abilities are important in the organization. However, they have some disadvantages. They may emphasize some characteristics and give short shrift to others (that is, not including them in assessments and appraisals) because they are not part of the company’s model. Competency models may not reflect the wide range of individual difference variables that may be important to leadership success. In addition, the competencies need to be revised periodically, since they reflect the competencies that are valued today, and other competencies may be needed in the future.
Next we describe specific leadership dimensions for assessment. We examine leadership potential, character, values, and motivation.
A range of characteristics may be relevant to a given executive position. Sometimes the demands of the position are not known or are likely to change, and the challenge of assessment is to identify potential. In this case, potential is “the ability to take advantage of developmental experiences that will be offered.”[7] Assessment for prediction of potential, however, cannot rely solely on measurements of past behaviors and experiences. The skills of an executive are developed over time, and as such, “end state” characteristics may not be present early in a manager’s career and would be hard to observe and use to predict later success.[8]
The ability to learn from experience may be seen by assessing such characteristics as curiosity about how people and things work, sense of adventure, hardiness (not threatened by criticism), bias toward action, accepting responsibility for learning and change, respecting differences, seeking and using feedback, and consistent growth. Leadership emergence is especially difficult to predict. Research has identified signs of leadership potential and emergence. For instance, elected group leaders speak more than other members and rate themselves higher on extraversion, and more skilled communicators are perceived by group members as more effective leaders.[9] Individuals evaluated more highly in selection interviews tend to be more nonverbally expressive and possess impression management skills.[10]
Assessments for selection determine whether specific candidates for executive positions will be a good fit with the organization, have consistent work performance over time, and demonstrate sound judgment when threatened or tempted or when doing the right thing may not be in their best interest.[11] Having superior knowledge, technical abilities, and skills does not mean that executives will live up to their promise. Character may be an important predictor of future success. Dimensions of character include self-determination, acceptance of responsibility, cooperativeness, self-transcendence (respect, caring, good citizenship—the polar opposite of self-interest), and good judgment (analysis, reflection, and enactment).[12] Values include characteristics such as loyalty, duty, respect, selfless service, honesty, integrity, personal courage.[13] The most important personal characteristics organizations seek are honesty, integrity, conscientiousness, interest in the job, and the “right general personality.”[14] Employers perceive the interview as the best way to assess these attributes, although there is a growing use of personality tests, measures of “emotional intelligence,” and “honesty and integrity” testing.[15]
Character may be assessed by modifying the standard executive assessment method of psychological inventories, interview protocols, and simulation activities.[16] Simulations are likely to be a useful method for assessing executive character. While well-crafted simulations and role playing can reflect the intensity of real-life pressures, behavioral indicators of an individual’s capacity to contain disruptive emotions are harder to come by. This is because nonverbal, contextual, and historical factors must be taken into account to recognize the ability to manage emotions and apply good reasoning while solving tough problems. Clearly, these are areas for the development and research of new simulations and measures. Character is one aspect of personality and does not replace the need for broader personality assessment in the executive selection process. Rather, it complements and broadens individual assessment methods.[17] We discuss methods of assessment in more detail next.
Another key characteristic for executive assessment is motivation to lead. Leadership motivation may be a function of the desire to achieve such extrinsic outcomes as wealth, status, and power as well as influence others to accomplish larger goals, such as the development and marketing of a product or helping others. The leader motive pattern (LMP) is high need for power, lower than average need for affiliation, and a moderate degree of activity inhibition—using power only for social or organization goals, not for personal needs. This pattern has been shown to be related to managerial advancement (r = .33).[18]
Another aspect of leadership motivation is entrepreneurship. Entrepreneurial leaders have ambition to start a new organization. They place emphasis on business growth, culture development, establishing credibility, and concern with the survival of the organization.[19] Having long-term goals and being high in self-efficacy have been found to be positively related to venture growth, such as sales revenue and employees, and these factors mediate the effects of passion, tenacity, and new resource skill on subsequent growth.[20]
To conclude this section on what is measured, overall, measures for leadership assessment stem from the process of studying the job itself (behaviors and outcomes expected), linking these job characteristics to individual characteristics and behaviors necessary to carry them out, and then developing measures of the individual characteristics and past behaviors. Competency models specify the abilities leaders need to be successful in the particular organization. To the extent that future-oriented competencies can be predicted now, they can be used to assess an individual’s leadership potential. Assessments of leadership potential can be used to identify people who should be in a development program that will prepare them for more responsible leadership positions.
As suggested previously, leadership assessment forces the organization to determine the dimensions of leadership that are important in the particular organizational context. Moreover, for assessment to be useful, it needs to recognize the changing nature of executive work (for example, the growing importance of knowledge management, the need for innovation and adaptation to technology, and stakeholders’ high expectations). Information about changes in competency requirements comes from leaders of the organization and industry who understand the context for leadership and what’s happening in the organization.
In general, the context presents requirements and challenges that need to be reflected in behaviors and characteristics for leadership assessment. Contextual factors may deal with market or economic forces driving the business, suggesting the need for leaders to have extensive financial knowledge and ability. Technological factors may be critical, which requires leaders who understand technology, not necessarily in the depth that an engineer would, but conceptually, in being able to comprehend and envision capabilities and how technologies tie together and generate market potential. Organizational and national culture and values will affect characteristics to be assessed—for instance, whether the organization has a long-term perspective that puts a premium on sustainability or a short-term profit motive. Selecting executives for international assignment requires assessing executives’ cultural awareness, cultural sensitivity, and ability to adapt to foreign situations.
Context influences leadership assessment by creating requirements and expectations for leadership characteristics and behaviors. Context needs to be incorporated in assessment methods, whether they are simulations, interviews, or performance reviews. For example, a business game used in an assessment can be designed to mirror the nature of the industry. A performance review that asks for ratings on what a leader could be expected to do can ask about expected behaviors under conditions and situations that are important to the company (for instance, developing new uses for existing products). Context also needs to be incorporated into uses for assessment data, for example, using assessment of general leadership characteristics to advise an individual about career prospects in a specific company or industry. For instance, information about an individual’s skills in organizing and planning can be compared to what type of organization and planning is needed in the current organization—maintain daily transactions or effect major transformations. Assessment that ignores context is likely to be irrelevant or of little use to making decisions or guiding individuals.
Measures and methods for leadership assessment include assessment for predicting leadership ability (that is, for selection, placement, and venture capital investment decisions), assessment for performance appraisal and review, including financial measures and balanced scorecards, and assessment for executive development. A review of all existing instruments and measurement techniques is beyond the scope of this chapter. Here we describe examples of frequently used measures that fit contemporary organizational needs.
A variety of assessment approaches are used, often in combination, for selecting executives. Psychologists are often called on by companies to conduct individual assessments of candidates. Such individual assessments are used for executive selection, promotion, development, and career counseling. The psychologist examines the job requirements and compiles a series of assessment methods that address the extent to which the candidates have the characteristics needed to meet expectations. However, the method suffers from a lack of reliability (agreement) among independent clinical assessments.[21]
The interview is the most commonly used method for executive assessment.[22] A structured interview process that asks the same core questions of all candidates will have higher validity than unstructured approaches. Structured interviews tend to have higher validity than unstructured interviews because structured interviews focus more on constructs that have a stronger relationship with job performance, such as applied mental skills (problem solving, judgment, decision making, critical thinking, planning, organizing) and social skills (for example, ability to function effectively in social situations).[23] Informal interviews will be more susceptible to legal challenges than structured approaches if only because two candidates may experience different interviews for the same position.[24] Structured interviews may ask about experiences and/or about what the candidate would do in different hypothetical situations.
Assessment center ratings can be used to identify people with senior management potential. An assessment center combines a number of exercises that might ask the assessed to write memos, prepare and give a presentation, or prioritize tasks and make decisions. Trained observers evaluate participants’ behaviors and products. While costly in time and money, they are popular tools due to their strong record of criterion-related validity, long-term predictive validity, and face validity. In other words, they tend to be related to actual job performance, predict future performance, and appear to be realistic to those assessed. However, their construct validity, that is, the extent to which they measure different elements of performance, has been called into question because ratings tend to be influenced more by specific exercises than by underlying performance dimensions. Guidelines for developing assessment centers provide ways to enhance their content, construct, and predictive validity.[25] These include adequate training to be sure that the assessors (observers and raters) understand differences in the elements of performance and how the same element of performance can be reflected in different situations (that is, exercises).
Cognitive tests measure the capacity to learn new skills. The Wechsler Adult Intelligence Scale and Watson-Glaser Critical Thinking Appraisal are frequently used tests. General mental ability measured by cognitive assessment is often the best single predictor of job performance across a range of leadership positions, with predictive validity higher on jobs of increasing job complexity.[26][27] However, cognitive ability does not always predict performance better than other measures, and it raises the potential for adverse impact against ethnic minority groups.[28] Therefore, cognitive ability tests should be used as a component of a larger assessment battery, not a standalone assessment method.
Measures of personality assess attitudes, motivation, and psychological character. Projective tests present ambiguous stimuli, such as pictures of people interacting, and then ask assessees to describe what is happening, under the assumption that personality structure affects how people perceive, organize, and interpret their experiences. Projective techniques, such as the Miner Sentence Completion Scale, have strong track records of validly predicting managerial and entrepreneurial motivation.[29] Assessees are asked to complete sentences such as “My primary goal is. . . .” The items are scored as positive, neutral, or negative based on guidelines and examples provided in the scoring guide. Different forms of the instrument measure different dimensions, such as professional commitment, acquiring knowledge, achievement motivation, attitude toward supervisors, and planning for the future.
Objective personality measures ask assessees questions about their preferences, attitudes, or values. Measures of the “Big 5” personality constructs assess conscientiousness, extraversion, agreeableness, emotional stability, and openness. All Big 5 personality dimensions have been found to be useful in predicting a range of specific performance criteria.[30] Conscientiousness, in particular, has been found to be valid for many jobs.[31] The validity of personality is generally modest, although it is larger when personality scales are selected based on a job analysis.
Financial performance and other objective indicators, such as sales and units produced at the organization and business unit level, are indirect measures of executives’ performance. Popular techniques recognize the multidimensional nature of executive performance and the importance of linking individual performance goals and measures to organization and business unit objectives—that is, alignment.
Financial metrics have historically played a very large role in assessing the performance of senior leaders. The role of financial measures in assessing leader performance is heightened because such measures are often closely linked to leader compensation. Although financial metrics have the appeal of appearing, at first glance, to be objective (at least when compared to assessing leader behaviors and competencies), a number of intractable dilemmas arise when using financial metrics to assess leader performance. Here we explore the issue of which financial measures might serve as appropriate indicators of leader performance.
Ideally, it would be desirable to identify measures that reflect the leader’s contribution and that make adjustments for factors that are beyond the leader’s control. Unfortunately, there is no single measure that accomplishes this goal. Several questions arise. One is whether measures should focus on individual or team (for example, corporate) performance. A second question concerns the relative emphasis on current or short-term performance versus longer-term measures. A third issue is the challenge of identifying a short-term financial measure that reflects the interests of shareholders and is aligned with long-term shareholder value. The core problem is that performance measures need to be selected so that when leaders pursue those measures, the firm’s objectives are advanced. What goes unmeasured will likely be ignored by senior leaders.
One measure of individual performance is the leader’s performance against a budget, profit plan, or other agreed-upon goals. But budgets, profit plans, and goals can be distorted when created by senior leaders (for example, in an attempt to make it easier to achieve the plan’s goals). At first glance, financial measures at the business unit or division level would seem to be useful measures of a senior leader’s performance. But focusing on business unit or division measures can create several problems. First, financial measures at the business unit level are of little use when several business units are highly interrelated (for example, thereby raising questions about how to allocate joint revenues and joint costs). Second, as Alfred Sloan noted more than 80 years ago at General Motors, leaders are too often inclined to focus narrowly on the performance (for example, profitability) of their own business unit or division and ignore how their efforts affect the performance of the firm as a whole. This question is especially important when tying compensation to leader performance. But linking rewards to team (for example, corporate) performance often creates a “line of sight” problem; leaders often fail to see a direct link between their behavior and corporate performance, and this lowers the motivational effect of team (corporate) rewards. In addition, leaders are often concerned with the possibility of social loafing or the “free rider” problem, when someone is rewarded because the leadership team as a whole performed well despite the poor performance of the individual. In sum, assessment of leader performance usually requires a mix of measures—some that assess individual performance and some that reflect team or corporate performance. Business unit or division performance measures are most appropriate in very decentralized firms where there is little interaction among business units or divisions. Corporate performance measures appear to be more appropriate in vertically integrated firms, where success depends on a high level of coordination among divisions.[32]
The management and popular business literature is filled with examples of leaders who have sacrificed a firm’s long-term value to maximize short-term profits (and thereby receive a bonus linked to short-term financial metrics). Stock options are often suggested as a solution to this dilemma because they are thought to align the interest of shareholders (the principals) with those of senior leaders (the agents). That is, stock price presumably reflects the extent to which investors value leaders’ actions. To the extent that stock markets assess the future consequences of leaders’ current actions, the firm’s stock price is a useful indicator of leader performance. However, stock prices are affected by many factors other than leader performance (for example, business conditions that affect an industry as a whole, a “bull” or “bear” market). One way to address this issue is to develop indices by which the performance of senior leaders is measured relative to the performance of comparable organizations in the same industry (thereby eliminating the effect of general market movements). Also, it appears that stock options sometimes influence leaders to avoid risky investments and decisions, whereas an ideal mix or set of performance measures will influence leaders (agents) to have the same risk preferences as owners or shareholders (principals).[33]
Short-term measures can be used to supplement stock price. Examples include earnings per share, pre-tax income, sales and backlog, or return on shareholders’ investment (equity). Short-term measures are often more directly affected by leaders’ actions than stock price. However, there is an imperfect association between accounting income and the long-term economic wealth of the firm. Moreover, senior leaders can take actions that increase reported income in the short-term but that decrease the firm’s long-term value (for example, failing to make investments in research and development (R&D), employee development, and maintenance that might increase the firm’s long-term value at the expense of short-term earnings; selling off assets whose market value is far in excess of book value).[34] In sum, the problem with accounting measures is the opposite of the problem with stock; accounting measures may be too easily affected (or manipulated) by the actions of senior leaders. One possible solution is for the corporate board (or its compensation committee), when assessing leader performance, to define earnings to exclude expenditures on long-term intangibles (such as R&D, employee development, and so on), thereby encouraging leaders to invest in such areas. But such practices are uncommon, perhaps because boards are not comfortable rewarding leaders based on one income figure while reporting a different figure to shareholders and creditors.
An alternative to focusing only on short-term accounting measures of performance is to assess leader performance by examining trends in accounting measures over several years (for example, growth in earnings per share (EPS) over a 3- to 5-year period). For example, firms sometimes award company stock to leaders who achieve a specified target in growth of EPS over a 5-year period.[35] Leader performance can also be assessed by examining the business unit’s operating results over a 3- to 5-year period. But when longer-term measures of a business unit’s performance are linked to leaders’ compensation, it can make it difficult to rotate leaders across business units.
In sum, when assessing leader performance using financial measures, it is generally advisable to include appropriate measures of current or short-term performance as well as measures that reflect long-term value. The most desirable measures are those that can be influenced by effective leadership actions and, at the same time, are not too easily manipulated by leaders willing to sacrifice long-term value for short-term profits (or by mere accounting manipulations).
An important trend in assessing leader performance is the increasing use of balanced scorecards. The balanced scorecard is based on the premise that focusing only on financial measures in performance management is insufficient because such measures are lag indicators (that is, they describe merely the outcomes of leaders’ past actions) and, as noted previously, can promote behavior that sacrifices long-term value for short-term performance.[36] Previous performance measurement systems that included nonfinancial measures generally used ad hoc collections of such measures rather than a comprehensive system of linked measurements. In contrast, the balanced scorecard selects a small number of measures that are directly linked to the firm’s strategy. In developing a balanced scorecard, firms must articulate a vision and ask, If the vision succeeds, how will we be different to our shareholders and customers and with respect to internal business processes and innovation and learning?
Balanced scorecards are also based on the premise that firms with different strategies require different measures. For example, senior investment bankers in a firm such as Goldman Sachs are valuable because of their knowledge about complex financial products and their ability to develop trust with sophisticated customers. But people with the same knowledge, experience, and capabilities would be of little value to a financial services company that emphasizes operational efficiency, low cost, and technology-based trading. That is, the value of an intangible asset depends on the firm’s strategy.
The mere use of financial and nonfinancial measures does not constitute a balanced scorecard. Instead, effective balanced scorecards are closely linked to the organization’s strategy so that people can understand the strategy by looking only at the scorecard and its strategy map. Balanced scorecards select a limited number of critical measures within each of four perspectives (financial, customer, internal processes, learning and innovation). Across all four perspectives there are usually only fifteen to twenty measures. Examples of financial measures include economic value added, return on capital employed, operating profit, cash flow, return on assets, project profitability, sales backlog, return on equity, and earnings per share. Examples of customer measures include customer retention, customer satisfaction (for example, from surveys), on-time delivery, share of key accounts’ purchases, market share, brand image, and the firm’s share in the most profitable segments. Examples of measures of internal business processes include quality, speed to market, rework, safety indices, complaint resolution time, cycle time, yield, and unit cost. Examples of learning and innovation measures include employee skill development, rate of improvement in key operational measures, number and quality of employee suggestions, development time for the next generation of products, percentage of sales from new products, employee turnover, and employee satisfaction (from surveys). The balanced scorecard communicates priorities to managers, employees, investors, and customers. It also helps facilitate change by forcing leaders to reach consensus on high-priority areas, increasing market orientation, creating a shared understanding throughout the firm concerning goals and what it takes to achieve them, linking compensation of senior leaders to the scorecard, and focusing improvement efforts in local units on key measures.
In general, several characteristics should be considered when selecting performance measures. Specifically, performance measures should be aligned with strategy, objective (for example, independently measured and verified), complete (capturing all relevant aspects of performance), responsive (leaders need to believe they can influence the measure), and linked to creating long-term value.[37] Also, the number of measures should be limited so that leaders can focus on a small number of variables that are critical to success. A properly implemented balanced scorecard satisfies these criteria.
Developmental assessment centers are similar in content and process to the assessment centers described earlier that have long been used for selection. Executive leadership requires a full spectrum of competencies needed in a constantly changing and increasingly competitive marketplace. Assessment exercises may represent a wide range of strategic, organizational, and interpersonal challenges. Behavior on these exercises coupled with results of the various tests may be used to predict versatility competencies, such as visionary thinking, shaping strategy, empowering others, influencing and negotiating, fostering open dialogue, and cross-functional capability measured by coworker ratings.[38]
There are a variety of measures of different approaches to leadership style and behavior. Often these are used in research on leadership, but they may also be used to assess the character and quality of leadership in an organization, give feedback to executives, and identify developmental needs. For instance, the Leader Behavior Description Questionnaire assesses leaders on dimensions of task and relationship orientation (initiating structure and showing consideration, respectively).[39] The Multifactor Leadership Questionnaire measures five factors: two facets of transactional leadership (Contingent Reward and Management-by-Exception) and three facets of transformational leadership (Charismatic Leadership, Individualized Consideration, and Intellectual Stimulation).[40]
Multisource (also called 360-degree) feedback surveys collect anonymous performance ratings from supervisors, subordinates, peers, and sometimes customers for comparison to self-ratings.[41] Sometimes data are collected only from subordinates, called upward feedback. The survey is often delivered electronically, distributed by e-mail that points the respondent to a Web site with the survey. Feedback reports usually contain item-by-item results with ratings averaged across respondents within groups, assuring anonymity of respondents by requiring a minimum number of respondents within the group of subordinates, peers, or customers. Recipients can tell, for example, how their subordinates or peers on average rated them on each item. Results may be averaged across items to reflect key performance dimensions and reduce the amount of feedback. Some reports also contain an index of the variation, or disagreement, among raters on the item or dimensions. Normative results—how other managers in the organization were seen by their raters on average—may also be presented to help the recipient interpret the results. Companies may administer the survey on a regular basis, perhaps annually or semi-annually, and provide recipients with data on change in performance ratings. A 360-degree survey may be administered for executives just prior to their attending a leadership development workshop and the results fed back to participants in order to pave the way for other assessment, coaching, and developmental goal setting. Another procedure, taking advantage of electronic media, allows executives to formulate their own survey whenever they want to assess how they are doing. Called just-in-time feedback, executives can select from a set of prewritten items or write their own and determine who receives the survey.
The popularity of 360 feedback has led some organizations to implement it haphazardly without adequate development, explanation of its purpose, or involvement of raters and ratees in writing items and formulating the administration and feedback process. Numerous guidelines have been offered about ways to make 360 feedback more effective.[42] Human resource experts argue that multisource feedback should be used for development alone, not for administrative decisions, such as compensation or promotion. However, companies that are not performing well sometimes turn to multisource feedback results to make difficult decisions about people or simply because the organization invested in the data.[43] The problem is that raters are likely to be biased (for instance, inflate their ratings) when they know the results will be used to make decisions about the people they are rating.
Multisource feedback should be viewed as part of a long-term performance management process, not a one-time event. The results should be used to set goals for performance improvement and development, perhaps in consultation with an executive coach or discussions with one’s supervisor or other coworkers. Also, changes in behavior and performance should be tracked over time, with change evident in subsequent multisource surveys and other objective and subjective indicators of performance.
As we stated earlier, assessment for selection requires periodic analysis of performance requirements. The same applies for assessment of executive development. Individual characteristics can be matched to developmental job components. The Developmental Challenge Profile (DCP) is one instrument used to assess the developmental components of managerial jobs.[44] The scale measures job transitions, task-related characteristics (creating change, high level of responsibility, and influencing others without authority), and obstacles faced on the job.
Assessment for development needs to incorporate job and organization evaluation as well as the measurement of individuals. We need frequent organizational assessment to determine changes in performance dimensions, which in turn will change what is assessed for evaluation and prediction. New methods for measuring individual performance and leadership style will be developed to counter biases and improve rating accuracy. The Internet poses opportunities for online measurement, tracking performance, and delivering assessment instruments to respondents. Information technology also allows customized questionnaires that probe more deeply when key questions raise issues. Executives can form and administer their own multisource feedback surveys when they feel they want others’ opinions about their performance. It doesn’t have to wait for the company’s formal survey that may only happen annually.
The value of leadership assessment can be viewed in terms of its economic value and issues of whether standards were followed in the development of the assessment method and whether the measurement methods have construct validity.
The financial impact of the assessment process can be evaluated. This is the fiscal return on the dollars expended as an organization’s investment in identifying, selecting, and developing leaders. Selection can be evaluated in terms of fit and performance. Measures of success may be intrinsic (career satisfaction, personality fit) or extrinsic (compensation level, rate of promotion, and perceptions of authority). Profits and cash flow are as much as 20 percent higher in companies with strategic HR practices. Assessments that help organizations hire the right executives for their organization, market, and goals will result in an increase in productivity.[45] Hiring a high-performing individual can generate gains of 120 percent in sales positions and 88 times their salaries.[46]
The financial value of assessment can be determined in large and small samples.[47] For large samples, say when a national organization uses an assessment center to hire entry-level managers in offices throughout the country, the payoff in dollars is a function of the product of quantity and quality minus costs. Quantity is the product of the average tenure of the executive in the position and the number of applicants selected. Quality is the product of the average score on the predictor (for instance, an assessment center overall score), the validity of the predictor (determined in a study of the correlation between the assessment center score and a performance measure), and the standard deviation of the performance measure. Cost is the expense of recruiting candidates and administering the assessment center. In small companies hiring few individuals, these key parameters can be estimated. Cumulative data across many studies can be collected to estimate validity. The standard deviation of performance can be estimated as 40 percent of mean salary.[48]
The value of assessment can be demonstrated by presenting evidence of construct validity. For individual executive assessments (for instance, a psychologist hired to evaluate candidates for a single executive position), a qualitative approach is needed to determine utility of the assessment method. Three steps are involved: (1) linking job tasks and individual characteristics, (2) linking individual characteristics to instruments of prediction, and (3) linking instruments of prediction to job tasks.[49] The first step is to generate information about the tasks to be done by the incumbent and the personal characteristics (knowledge, skills, abilities, and other characteristics—KSAOs) required to do these tasks. The idea is to identify the common constructs that underlie the job and the individual. The next step is to link the KSAOs to prediction instruments, such as work samples, attitude and ability tests, and interviews to show that the KSAOs needed are measured. Then individual components of these predictors (for example, test items or structured situations posed during an interview) should be linked to job tasks in order to show that the constructs needed for effective job performance are actually measured by the assessment methods. Several subject matter experts may be asked to do this to ensure reliability of the judgment.
Leadership assessment occurs within the context of specific conditions, requirements, and issues. The following case illustrates how leader assessment center and 360-degree survey feedback are used along with consultation to select and improve an executive’s performance.
Melinda was promoted to head the sales department of a large manufacturer of consumer products. She was selected after an intensive search and selection process that included putting candidates through a specially designed assessment center involving case exercises, structured interviews, and personality tests. At the end of Melinda’s first month, her sales figures were not impressive. She met individually with the sales directors who reported to her to review their goals, operational plans, and metrics. She inserted some new sales strategies and methods of gaining product visibility. However, the first month turned into the first quarter with negligible improvement in sales. Realizing that her suggestions were not working, Melinda set concrete demands and ever-higher expectations on her management team. Some of the sales directors contacted the CEO to complain. Unaware of the discontent among her staff, Melinda focused on making the numbers. She made progress, too. Not incredible progress, but she gained 2 percentage points in the second quarter. Still, morale in Melinda’s unit was suffering. Her sales directors alleged she used intimidation and verbal abuse to drive their performance.
The CEO consulted with the vice president (VP) of human resources, who met with Melinda to review the situation. The VP suggested that they collect some additional information. “Maybe” he said, “it’s just a style thing.” Melinda acquiesced, and the VP arranged for a 360-degree process. The CEO, her direct reports (the regional sales directors), and her peers (the other VPs) provided ratings. At first glance, the findings seemed conclusive. Melinda thought well of herself, but her subordinates rated her low on many performance dimensions, for instance, low ratings in providing resources, serving as a coach, offering encouragement, listening, being open to new ideas, and setting reasonable goals (ratings were made on 5-point scales, and dimension means from the subordinate ratings ranged between 1.0 and 2.5). On the other hand, the CEO and her peers found her to be a cooperative colleague who understood the changing needs of the business. An executive coach delivered the results. Melinda listened, perused the data, focused on her subordinates’ ratings, and with cunning insight concluded, “So they hate me. Is this supposed to be a revelation?” The coach pressed Melinda for a finer-grained analysis of the data. After several hours of discussion, Melinda recognized that the sales directors viewed her as stronger in diagnosing market conditions, refining strategy, setting goals, and tracking performance and weaker in listening, involving them in decision making, and being open to new ideas. She talked to the coach about why she felt the difficult market conditions meant she had to maintain tight control over sales strategy, and they discussed ways she could be more participative and take advantage of her directors’ vast experience and detailed knowledge of their own territories.
Melinda made a concerted effort to change by holding brainstorming and strategy sessions with the sales directors, asking them to set goals and timelines, and then holding meetings to examine their degree of success, determine ways they could improve, and revise their goals. This wasn’t always easy for Melinda. She had to force herself to be less controlling and more trusting. Sometimes she caught herself “telling” rather than “asking,” but she was more attentive to what she was saying and how others were reacting to her. Also, when she became directive, her sales managers would point this out, and they would laugh about it. Still, Melinda and her sales managers realized there were times when she had to make a decision, and that she was ultimately accountable and it was her prerogative to call the shots even if they disagreed, and this happened occasionally. When it did, the sales managers were more accepting than they were initially because Melinda had taken their views into account.
Melinda’s promotion and coaching coincided with the company’s implementation of a balanced scorecard to help department heads set and track goals in line with corporate interests. She was able to use this process with her sales managers as a basis for discussion and ongoing evaluation. The process was also a tool for assessing Melinda’s and other department heads’ success at developing their people, involving them in formulating and implementing strategy at the local level, and seeing how their success contributed to performance metrics at the department, division, and ultimately the corporate level.
This case shows how assessment center results, 360-degree feedback data, coaching, and ongoing monitoring of objective performance were all components of leadership assessment and development. An effort that began with individual assessment was supported by a companywide balanced scorecard process for leadership goal setting and performance evaluation.
We began this chapter by emphasizing that leadership assessment entails measuring process and outcomes and that assessment in itself is a process. Assessment occurs on different levels—organization, team, and individual. Also, it measures multiple dimensions—financial, personal, and interpersonal. Measures alone have little meaning unless they are compared to goals, standards, or norms over time.
Leadership assessment is used for predicting performance (selection and placement), evaluating performance (holding executives accountable and compensating them), and diagnosing performance gaps and setting directions for improvement and career development. The assessment process forces attention to what is measured and raises the question, Is this important? The assessment process and results reflect the culture of the organization and also can be an intervention to alter the organization’s culture.
Assessment requires determining important components of job behaviors, performance, and outcomes, linking these to organizational and individual characteristics, and identifying measures of these characteristics. Methods of assessment for selection, placement, and investment decisions include interviews, assessment centers, cognitive tests, and personality inventories. Methods of assessment for appraisal entail measuring multiple dimensions of performance, including indicators of financial conditions, customer interactions and outcomes, internal work processes, and participation in learning and creation of innovations. A balanced scorecard of these measures aligns organization, team, and individual goals with associated performance indicators. Assessment methods for executive development include assessment centers, measures of leadership style, and multisource (360-degree) feedback surveys. Assessment methods themselves should be assessed to determine that they were developed using professional guidelines, have construct validity, and create economic value. The case demonstrated assessment for leader selection, performance improvement, and development.
We conclude with ten recommendations for leadership assessment:
Determine the purpose of assessment. Is it selection, performance evaluation, and/or development?
Given the purpose, what do you want to measure? Alternatives include measures of current performance, potential for future performance, capabilities, knowledge, motivation, and character (values).
Examine how situational conditions determine what elements of leadership are important. Consider how to incorporate context into the assessments, for instance, observing performance in simulated environments that are similar to actual work settings.
Determine organizational goals, current conditions, and anticipated shifts in your organization and environment that influence leadership behaviors to be assessed.
Consider how to measure both performance processes (how leaders go about doing their jobs) and outcomes (results).
Determine whether your focus is on the individual, team, and/or organizational level. Characteristics assessed may be personal (abilities, skills, knowledge, motivation, and values), interpersonal (group performance, cross-enterprise partnerships), or financial (for example, business unit profitability).
Identify or develop methods of assessment, including assessment centers, personality measures, interviews, and 360-degree feedback surveys.
Use goals, standards, and norms to provide a basis for interpreting results. That is, interpret assessment results in terms of what is important in the organization or industry.
Use balanced scorecards to align organization, team, and individual goals with a range of performance measures.
Show the value of your assessment process and results—both the economic value to the organization (for example, how assessment is used to affect the bottom line of the organization) and evidence that the constructs measured are valid (that is, the discrete elements of performance that are relevant to the organization and leadership in general).