In this chapter, we will cover how to plan and organize a CMMI appraisal for your group or organization, how to prepare and guide your teams for final CMMI appraisal, and how final appraisal activities are conducted. By the end of this chapter, we will have touched on most of the things which are required to plan for a successful CMMI Implementation. Specifically, this chapter will cover the following points:
Identify the objective, constraints, and scope of an Appraisal
Identify/develop an appraisal plan
Prepare and train the team on appraisal methods
Plan and prepare for appraisal conduct
Let’s start with the first point of planning for a CMMI appraisal.
6.1 Identify the Objectives, Constraints, and Scope of an Appraisal
For most readers, this point is more like a generic practice. Whenever any project gets started, the first task is to analyze the requirements, and after that, to identify the objectives and scope.
6.1.1 Analyze the Requirements
For CMMI or any other appraisals, analyzing the requirements can be the most important process because it can be the foundation of success or failure for the complete appraisal. Most appraisal downfalls are due to a lack of requirements gathering.
From a CMMI perspective, the purpose is to understand the business needs of the organization for which the appraisal need to be conducted. The appraisal team leader of the lead appraiser (LA) will collect information and help the business head, unit head, or sometimes, the sponsor.
Various tools and techniques can be used for analyzing the requirements, such as interviewing the sponsor to get the needed information. A formal consultation is very important; sometimes, a series of meetings need to be conducted with the stakeholders for their consensus on the business needs, which can be done using a SCAMPI A appraisal.
6.1.2 Identify the Appraisal Objectives
With the series of meetings conducted, the business needs have been identified. In general, CMMI is requested when the organization or the business unit wants to accelerate its process improvement, which includes one or more of these parameters:
Improving quality
Reducing efforts and/or cost
Decreasing time to market
There can be a variety of reasons for an organization to conduct a CMMI appraisal. For example, it can be driven by business objectives like the following:
Documenting the benchmark that reflects business process improvements
Identifying the potential risk areas of an organization which may impact the performance of an organization
Supporting scientific decision making based on data and analytics
Identifying the strength, weakness, opportunity and threat (SWOT) of the organization or business unit
As per the CMMI handbook, the appraisal team leader will ensure the following:
The sponsor and relevant stakeholders are identified
The business objectives have been documented (these will be provided by the sponsor)
Whether the appraisal objectives have been aligned with the business objectives
The appraisal usage mode has been identified (i.e., internal process improvement, supplier selection, or process monitoring)
Note
It is very important to schedule one meeting/discussion between the appraisal team leader and the CMMI sponsor (within the organization).
6.1.3 Identify Appraisal Constraints
The constraints can also be determined by scheduling the discussion between the lead appraiser and the sponsor and/or business head. The business/unit head must communicate whether there are any limitations which require negotiation with the lead appraiser.
Some constraints, like cost and schedules, can be identified at early stages of the appraisal, which are at a high level only. Examples of some basic constraints include the following:
We need this Appraisal to be done in Q3 or Q2.
You cannot use more than ten people on the team.
Management review can be done only via a video conference, teleconference, or through similar technology.
6.1.4 Determining Appraisal Scope
The scope of the appraisal consists of the appraisal reference model (i.e., CMMI DEV, CMMI Services, or CMMI Acquisition) and the scope of the organization/unit that would be assessed during the appraisal.
The organization must ensure that the scope of the reference model is determined and documented early in the planning process. It can do so by using the staged representation or the continuous representation.
When we say organizational unit, we are referring to a subgroup within the organization for which appraisal activities will be carried out.
Note
An organization may include the whole organization or a business unit (it could be small or big) within the organization, or it may select only a few development projects for the CMMI implementation and appraisal/ratings.
As per the CMMI handbook, the following are the required practices for identifying the scope of the appraisal. The lead appraiser, along with the appraisal sponsor and/or the business/unit head, shall do the following:
Identify the scope of reference model and representation that will be used for the appraisal
Identify the organizational unit and scope to be assessed
Identify team members who will participate in the appraisal
In the appraisal scope, we have to identify sample projects which could be assessed, as well as the support functions (i.e., SEPG, QA, and training). In the project scope, we must also include the organizational unit size (i.e., the total number of people currently working within the organizational unit and the total projects which are currently active).
As per the CMMI handbook, the sample projects and support functions have to showcase the following during the appraisal:
Primary projects: Objective evidence for each process area under the scope of appraisal
Secondary projects: Objective evidence for one or more process areas under the scope of appraisal
Support functions: Objective evidence for practices under the scope of appraisal
6.2 Develop Appraisal Plan
Like any other plan, the appraisal plan is very important. It will act as a guide for the execution of appraisal activities. The lead appraiser will document the appraisal plan will share his/her review with the sponsor. A tool for developing the appraisal plan could be an appraisal plan template.
An appraisal plan may include the following appraisal planning activities:
Tailoring method
Resources required
Cost and schedule
Logistics
Risks
Appraisal plan commitment
Let’s drill down in detail on each activity required for the development of an appraisal plan, beginning with the tailoring method.
6.2.1 The Tailoring Method
In concert with SCAMPI A, some of the tailoring which could be allowed is as follows:
Depending upon the choices, there could be options within the required practices
Parameters could be allowed to vary within the limits
The SAM (Supplier Agreement Management) process area is optional, and it can be mentioned as out of scope for the organization or the business unit going for an appraisal
For SCAMPI A, the tailoring activities should be performed carefully because what should be tailored or made out of scope as per the model is very important for the business unit to be appraised. For example, consider a case where an organization wants the SAM process area to be out of scope for the appraisal as currently there is no outsourcing work, but the company’s senior management has been in discussions to outsource some of its work to vendors in the near future (i.e., the SAM process area). If the CMMI team is not aware of this update from management, then there is a chance that the team will make the wrong tailoring selection. To avoid such scenarios, the CMMI team should be in constant touch with its senior management before making any decisions for the appraisal activity.
The CMMI guidelines expect the lead appraiser to do the following:
Perform a review of each activity and select tailoring options accordingly
Ensure that the tailoring decisions taken are appropriate to meet the appraisal objectives and constraints
Ensure that the decisions made for making particular tailoring options are documented
Note
An organization must ensure that selected method tailorings and their implementation do not violate SCAMPI A Required Practices.
6.2.2 Identify Needed Resources
When conducting the appraisal, identify and estimate the resources required. Resource categories to keep in mind for the appraisal include human resources, infrastructure facilities, tools, and so on.
We have to ensure these resources are accounted for in advance because this will help the appraisal activities run more smoothly.
As per the CMMI guidelines, the lead appraiser will do the following:
Ensure that the appraisal team members are identified by the business unit which needs to be appraised, as these appraisal team members need to participate in the final appraisal with the lead appraiser. These members should have sufficient experience in project management and engineering (i.e., SDLC)
Ensure that the participants (i.e., functional area representatives (FAR)) from various project teams are identified by the business unit; FAR areas include requirements, design, coding and unit testing, and integration and testing
Ensure that equipment and facilities have been arranged by the business unit
Ensure that resource decisions are documented in the appraisal plan
Note
If appraisal team members (ATMs) are not available for the SCAMPI A appraisal from within the organization, then, as per CMMI guidelines, the business unit is allowed to hire ATMs from outside the organization for the appraisal duration.
Also, during the appraisal planning, it’s important to identify the projects to be appraised, along with names of the people who are going to participate as team members in the appraisal.
You also need to identify the seating capacity and environment of the rooms which will be used by the teams for conducting the appraisal activities. Please be aware that resource planning is not limited to the personnel only; it is also includes a projector, laptop, or video-conferencing equipment (for any teams or senior members who are not present when the business unit is to be appraised).
6.2.3 Determine the Cost and Schedule
After estimating the resource requirements, you need to estimate the cost and schedule; as per the CMMI guidelines, there are required practices for estimating the cost and schedule. The lead appraiser will do the following:
Estimate key appraisal events for the required duration in order to form a detailed schedule
Estimates the effort of participating members in the appraisal
Estimates the cost for using the business unit facilities and equipment, as appropriate
Estimate the cost for any travel, lodging, and meals, as required at the time of appraisal
Document the detailed appraisal plan
6.2.4 Manage Logistics
The next step is to manage the logistical plan which must take in the details and should be documented. The team leader must document logistical schedule and dependencies if any. They must maintain the communication channel for providing status and assign responsibilities to track logistical issues.
To make the logistical planning more effective, you must plan for the following details:
Hotels or guest houses for the members and lead appraiser, if travel is required to perform the appraisal at different locations or even outside the country
Any workstation support
The arrangement of food and beverages
Good Internet connections and other accessible equipment that might be required during the appraisal
6.2.5 Document and Manage Risk
As with any other project risks, it is very important to plan and manage risks proactively for the CMMI appraisal. The lead appraiser will document the identified risks and associated mitigation plans. The lead appraiser will also communicate these risks and their associated mitigation plans to the CMMI sponsor, CMMI team leader, and the ATMs.
6.2.5.1 Obtain Commitments to the Appraisal Plan
This is the final step for developing an appraisal plan, where the formal commitment is obtained from the sponsor. The appraisal plan constitutes an agreement between sponsor and the lead appraiser.
The lead appraiser will do the following:
Ensure that the appraisal plan is reviewed with the sponsor and that the sponsor’s approval is secured
Ensure that the appraisal plan is also reviewed with the relevant stakeholders for their review and acceptance
The lead appraiser and the sponsor have to sign the appraisal plan to secure the commitment towards the plan.
6.3 Prepare and Train the Team on Appraisal Methods
Once the appraisal plan is prepared and the commitment is obtained from the sponsor, it’s time to ensure that an experienced and qualified team is ready to execute the appraisal process.
Approximately one month before the scheduled appraisal date, the lead appraiser will conduct the mandatory four day training session for the ATMs. Of the four total days, the CMMI intro session will be conducted for the first three days; and on the last day, the ATMs will be taught how to conduct the appraisal. This will help the ATMs to be clear on the concepts of CMMI, so that they will be ready to help the lead appraiser.
Every lead appraiser’s teaching style is different; hence, ATM members can undergo a somewhat different appraisal experience if they get the chance to participate in more than one appraisal. However, if the lead appraiser is the same during the second CMMI appraisal, then the ATMs will have to wait for a new lead appraiser.
Note
For some ATM members, it could be their second CMMI appraisal; in such a scenario, they don’t need to attend the CMMI intro training again; instead, they need to attend only the one day training on how to conduct the appraisal, as attendance on this day is mandated by the CMMI Institute. There is one catch here, however; this scenario won’t work if the version of CMMI model has changed since the last time that ATM member underwent the intro training.
6.4 Plan and Prepare for Appraisal Conduct
Here, the motive is to ensure readiness for conducting the appraisal. This readiness includes the availability of filled PIIDs which show the mapping of the CMMI model practices against the objective evidence from projects and support functions, as well as the availability of appraisal team members, updated risk status, and the associated mitigation plans.
Upon assessment, the sponsor and the lead appraiser will decide whether the appraisal plan will be executed as defined or need to be re-planned. They will also have to decide whether any corrective actions need to be taken against the plan to ensure that the plan is successfully achieved.
The lead appraiser will perform the following steps to check the readiness of the appraisal:
Check the availability of objective evidence for each practice in the process area, as per the defined appraisal scope
Ascertain the readiness of ATMs to conduct the appraisal
Determine the availability and readiness of all the logistics (i.e., the infrastructure facilities, equipment, etc.)
Review the status of appraisal risks and their impact
Note
The readiness review can be assessed on-site, off-site, by video- or teleconference, or in a combined manner.
There will always be more than one readiness review, as there could be leftover gaps which the teams will need to work on and close.
6.5 Conduct the Final Appraisal
On the first day of the final appraisal, the opening meeting will be held in which the lead appraiser will present the appraisal plan to all the teams and the sponsor. Also, the lead appraiser will cover how the appraisal plan will be executed and the activities which are planned each day (i.e., documents review, interviews with different teams, etc.).
After the opening meeting, the lead appraiser will review the documents with the appraisal team members. Once this exercise is finished, the interview sessions will start the next day; usually, these sessions start with the project management interview, followed by the project team members, including the SEPG, QA, training, and management members (i.e., it could include the program manager).
Once all the interviews are finished, the lead appraiser will, along with the appraisal team members, consolidate the findings and assess whether there are any weakness/improvement points against any practice.
During this exercise, if there are any queries to be asked of a project manager or any other team member who has already been interviewed, then that person can be called back to the interview room for clarification. For example, during the interview session, a project manager might communicate that status reports are being shared with the program manager, but the objective evidence provided to the lead appraiser and appraisal members does not support this assertion. In this scenario, the project manager would be called again to show some evidences that the status reports have been communicated to the management. If evidence is available, then no weakness would be reported; however, there is a possibility that the improvement suggestion could be noted.
After all the evidence collection, interview sessions, and clarifications, the lead appraiser and appraisal team members will start rating each process area, specific goals, and/or generic goals.
If all the specific and generic goals are fully or largely implemented and satisfied, then only the overall organizational unit would be rated as satisfied. Once all the process areas have been rated, then the next step would be to prepare the findings presentation. All appraisal members will prepare their findings/slides, and these will be reviewed with the lead appraiser.
Once the slides are reviewed, they will be merged together to form the preliminary findings. These findings will be presented first to all the team members who were interviewed, so as to ensure that any finding which is noted is not incorrect.
After the preliminary findings session on the last day, the final findings presentation is done for all of the organizational unit and team members who participated. The findings are comprised of both strengths and weaknesses. Strengths document things we have to carry forward in the future, and weaknesses document areas we need to look at.
When all the findings are presented, the final slide will tell you the result, whether you are successfully appraised for CMMI Level 3 or CMMI Level 5, as per the appraisal scope.
6.6 Summary
In this chapter, we covered planning for identifying the scope, constraints, and objectives of the appraisal. We have also covered how to develop the appraisal plan, and clarified the roles and responsibilities of the appraisal team leader and the lead appraiser. This chapter also discussed how to prepare and plan for the appraisal.