Chapter 10

UNDERSTANDING PROJECT PERFORMANCE MANAGEMENT

Tracking investment dollars is a familiar activity in the federal environment. Yet it remains unclear how much value the federal government and American taxpayers receive from the massive investment in projects. The performance management framework that operates in the federal environment can shed some light on this value.

PERFORMANCE MANAGEMENT IN THE FEDERAL GOVERNMENT

The federal government uses a variety of tools to manage performance, including E-300s, GAO’s high-risk list, IT scorecards, variance reports, and corrective action plans. Cost, schedule, and performance goals must be established for all major acquisitions, and agencies must achieve, on average, 90 percent of those goals. OMB Circular A-11, Part 7, provides a critical source of guidance in its Capital Programming Guide supplement, which lays out the important acquisition steps and identifies earned value as a method of measuring project progress. In total, these systemic changes to performance management have brought about improvements to the methods agencies use to establish strategic goals, work with contractors, use performance data, and manage projects.

Performance-based management involves establishing, managing, measuring, and evaluating performance data. Project performance must link into the overall performance management program, and the organization must ensure that projects can be measured individually and in tandem with other projects in the portfolio. This is a significant challenge for many agencies.

Success in project management in the federal environment involves recognizing the following:

   •   Performance management is multidimensional, requiring collaboration and coordination between acquisition, finance, operations, policy and planning, human resources, oversight, and other agency divisions.

   •   Performance management involves all levels of the government, from the single project to agency executives, OMB, and Congress.

   •   Performance management is predicated on the effective implementation of earned value management.

   •   At the heart of effective performance management is good data, including information about the reliability of the data.

   •   EVM and project management systems have evolved to the point where effective project reporting can be accomplished by agencies with limited project management expertise.

The primary focus of the performance-based management strategy has been on major IT systems and the use of earned value management to monitor progress. While EVM and its predecessors have been in use in the federal government since the 1960s, EVM is only now gaining widespread traction across both defense and civilian agencies. The government continues to develop the performance management capability to ensure that capital investments are performing well.

IMPROVING PROJECT PERFORMANCE

In the context of the broader federal performance measurement system, project performance management offers opportunities for agencies to improve project performance management across three phases of a project:

   •   Establish project performance measures. Performance systems are developed at the organizational level, and baselines are put into place at the project level.

   •   Manage project performance. Effective performance management strategies help establish a rigor within the project team.

   •   Oversee project performance. An independent oversight board examines the project to verify that performance is on target and that the underlying systems and data are adequate.

Each of these phases lends itself to an overall system of project performance management. Each fits into the larger flow of performance data that flows upward from project to agency to government-wide performance scorecards, and ultimately to congressional hearings.

Establishing Project Performance Measures

As the old adage goes, “If you can’t measure it, you can’t move it.” Performance measurement systems should enable a project manager to closely monitor the project and enact changes. Establishing performance measures in a project environment involves creating measures at the individual, work team, and project levels, as well as creating policies and the operational framework to ensure that high-quality performance data is generated. To ensure a successful project performance measurement program, the sponsor and key agency leadership need to consider three key areas that will enable a full-fledged performance management system: investment review boards, EVM, and non-EVM performance criteria.

Investment Review Boards

At the organizational level, the agency must create an environment conducive to high performance. For example, in IT projects, OMB has recommended the use of an IT investment management (ITIM) maturity framework, whereby the agency creates one or more investment review boards, or IRBs, to oversee the development of IT investments.

The IRB is a common mechanism in agencies for bringing together IT and business executives in a process to define and understand the investment. The investment has to be clearly understood in terms of both its costs and its benefits by all stakeholders. Under this framework, IRBs consider new projects and reevaluate prior investments to ensure they are on track. To do this, IRBs must establish new performance baselines and evaluate existing project performance data.

The ITIM framework is not without challenges. IRBs must have the right people involved in the process. To be effective, senior directors from across the organization must participate in the process. Because the IRB is a decision-making body, delegating IRB meetings to lower-level staff is not a good practice. These boards must also have the right technical and nontechnical staff in attendance. The complexity of these investments, and their success, sometimes rests upon arcane technical details that will make or break the investment.

The IRB must also have the right processes in place for considering new investments, reviewing existing investments, and assessing performance. When those processes are not in place, investments can be ignored or left unanalyzed. In addition, when corrective actions are called for, the IRB must be prepared to track the completion of those actions. GAO recommends that agencies “strengthen and expand the board’s oversight responsibilities for underperforming projects.…”¹ These challenges are longstanding, but clearly the ITIM framework, and the IRBs, allow for continuous evaluation of project performance data at the appropriate levels of the agency.

Earned Value Management

At the heart of performance management in the federal government is earned value, based on ANSI/EIA standard 748 (2007). Earned value is a project management concept whereby the value of the work completed in a given period is compared with the planned value of the work and the actual value of the work. Cost and schedule data are used to help understand the true performance of a project. For example, if a project team completed a new $2 million parking lot at an actual cost of $3 million, there would be a –$1 million cost variance.

Schedule data also informs the performance data and uses the earned value of work for a given time period as compared with what was expected to be completed at the end of the time period. For example, if, at the end of the month, the project team has completed $1 million of the work, but was budgeted to have completed $1.1 million, the project would have a –$100,000 schedule variance.

The objective is to understand the planned value of the work at any given point—the earned value. This level of information provides executives and stakeholders across the government a look into the status and overall health of the project.

The DOD 2006 Earned Value Management Implementation Guide provides details on what EVM can accomplish.² The guide states that an EVM system (EVMS) allows for the following:

   •   Planning of all work scope for the program to completion

   •   Assignment of authority and responsibility at the work performance level

   •   Integration of the cost, schedule, and technical aspects of the work into a detailed baseline plan

   •   Objective measurement of progress (earned value) at the work performance level

   •   Accumulation and assignment of actual costs

   •   Analysis of variances from plans

   •   Summarization and reporting of performance data to higher levels of management for action

   •   Forecasting of achievement of milestones and completion of contract events

   •   Forecasting of final contract costs

   •   Disciplined baseline maintenance and incorporation of baseline revisions in a timely manner.

On the civilian side of the federal government, only a handful of agencies have vigorously embraced EVM for many years, including NASA and DOE. These agencies adopted the American National Standards Institute/Electronics Industries Alliance national standard for EVM, called ANSI/EIA-748, which provides the “industry process for the use of EVMS including integration of program scope, schedule and cost objectives, establishment of a baseline plan and accomplishment of program objectives, and use of earned value techniques for performance measurement during the execution of a program.”³

Federal management and acquisition teams have begun to focus on EVM in recent years. EVM can be used as a stand-alone system or in conjunction with other measurement methods. It can be applied at various levels, from individual projects all the way up to cabinet-level department initiatives.

Earned value relies on extensive data—including direct labor data, materials cost data, schedule data, indirect cost data, and contract data—from a wide variety of sources. The systems that enable earned value are complex, and in the case of contractors, must be certified by the federal government as appropriate for use as an EVM tool. An EVMS must be aligned with the organization and its resources to ensure the proper allocation of work. The organizational breakdown structure (OBS) is essential to linking work with specific organizational units. The resource breakdown structure (RBS) accomplishes a similar objective, but with project resources.

The first step in defining an effective EVMS is to overlay the WBS, OBS, and RBS. In other words, effort should be catalogued by its WBS reference number, assigned organization, and actual resources used. The project manager will thereby be able to track performance across those various elements.

A fully compliant system is no small feat, as the guidelines address organizational structure; accounting, budgeting, and finance; project management; and human resources. Additionally, OMB has established “key practices” supporting systems acquisition programs, summarized into three areas:

1.    Establish a comprehensive EVM system.

2.    Ensure that the data resulting from the EVM system are reliable.

3.    Ensure that the program management team is using earned value data for decision-making purposes.

While the three areas are interdependent and each area is important, the third is the most critical for the agency. EVM helps ensure that better decisions are made through the integration of performance data.

Non-EVM Performance Criteria

EVM provides essential performance data; however, the project also maintains other performance criteria that are important to evaluating the overall success of the effort. For example, safety performance criteria are not necessarily embedded within the EVM calculation, although injuries and work slowdowns due to unsafe conditions will impact project performance. The team may have a performance criterion that states “This project will incur zero accidents until its completion.” This is a goal that is separate and apart from the performance of the project, but it should nonetheless be used to evaluate the overall success of the effort.

Another example is training. Ensuring that all project staff receive proper training can be tracked as a separate performance measure even though it will impact overall project performance. Training will probably be built into the schedule, so if it is not completed on time, it will directly impact the EVM calculation. Viewed separately, untrained staff members lead to poor-quality deliverables and rework, so clearly training impacts performance. Moreover, the organization will not be prepared to maintain and operate the final deliverables if staff members have not received the proper training. So tracking training goals as a separate performance measure provides a meaningful way to realize the outcomes of the project.

If a project manager maintains a zero variance for cost and schedule, but does so because the project is hiring lower-quality labor and not providing safety equipment or training, the project’s risk profile will increase. Non-EVM performance criteria should drive decisions about tasks, budgets, and time frames. These are important filters for understanding how projects are faring.

Managing Project Performance

Performance management goes hand in hand with project management. The project team executes on the plan and generates the performance results. Invariably, problems that degrade the performance of the project arise. For example, resource shortages may occur or the cost of materials may increase. Sometimes these issues were planned for and alternatives can be implemented with minimal disruption. Other times these circumstances occur without foreknowledge and the team must scramble to get back on track. In either case, an effective management strategy must be followed to ensure that project performance is monitored and corrected as necessary.

Performance management is cyclical, with each turn presenting opportunities for improvement.

These opportunities include the following:

   •   Maintain accountability. An effective project performance strategy begins with accountability. The project manager must know his or her role and responsibilities, as should the rest of the team. Individuals should be accountable for specific outcomes and results. Accountability is the precursor of ownership, which is a critical element in performance. In short, today’s knowledge workers must care or performance will likely degrade.

   •   Actively monitor. A performance management strategy also requires an effective monitoring plan. Performance should be monitored and evaluated regularly through performance reports, post-implementation reviews, and on-site reviews. Yet, performance issues do not always come from reports, but often arise from direct observations and interactions with staff. The project team needs to be able to discuss performance issues honestly without defensiveness or fear of retribution.

   •   Regularly measure. Performance measurement is the mechanism for collecting, reporting, analyzing, and evaluating project performance data. The data used in many federal systems has been noted as suspect, due to poor collection processes.

   •   Take corrective action. Next, the team needs the ability to respond to performance issues. The team should be empowered to act on the performance data within the parameters of its responsibilities. For some federal projects, this may mean implementing contingency plans. In other instances, it may require executing scenario-based training. Corrective action should address the core issues related to performance.

While many different strategies will be effective in managing performance, this straightforward set of steps can enable federal managers to positively impact their projects.

Overseeing Project Performance

The federal government has invested in tracking tools designed to oversee projects and programs that aren’t meeting performance goals. Early detection and correction of problems, while the project is underway, is becoming a best practice in federal projects. In recent years, increased emphasis has been placed on measuring projects more consistently. OMB retains much of the responsibility for implementing programs that will result in improvements. For example, OMB now requires that all large projects use an earned value management system. On the congressional side of the measurement spectrum, GAO provides oversight and accountability in its reviews of individual programs and agency results.

OMB’s role in project performance has been increasing in recent years, through the use of such tools as the Program Assessment Rating Tool, Exhibit 300, and performance dashboards. These tools are evolving into effective instruments for tracking and improving project performance. Further, OMB has the authority to impose consequences if agencies do not deliver.

Not only does OMB’s scorecard include traditional data elements, but it posts photographs of the agency CIOs. The intent of putting a literal face to senior management is to drive accountability throughout the entire agency.

OMB leadership has clearly come to realize that project performance management is an ongoing process. Project evaluation and measurement, both during and after projects, are conducted not just to identify and correct errors but also to determine and evaluate what worked and how successful methods might be replicated.

Federal project managers and their contractor counterparts are faced with a dilemma of divergence. On the one hand, projects are rapidly increasing in complexity. From legislative and regulatory requirements to technical challenges, the level of project complexity has never been greater. On the other hand, stakeholders are demanding more and more for their tax dollars. Citizens and lawmakers alike understand that projects must deliver value, and they have the tools to be heard.

Project performance management starts at the agency level. To be successful in the federal environment, agency leadership would do well to understand how EVM works. Pursuing performance management takes courage and conviction because it will invariably show areas where performance can be improved. Yet, it must be done. As projects take up more time, dollars, and operational space, it is important to build the proper support structure to be able to manage new initiatives as effectively as possible.

NOTE

1.    U.S. Government Accountability Office, “Federal Agencies Need to Strengthen Investment Board Oversight of Poorly Planned and Performing Projects,” GAO Report 09-566, June 2007, p.7.

2.    U.S. Department of Defense, Earned Value Management Implementation Guide, October 2006, p. 3.

3.    Federal CIO Council, “A Framework for Developing Earned Value Management Systems (EVMS) Policy for Information Technology Projects,” December 5, 2005.

4.    Federal CIO Council.