Purpose

The purpose of the change evaluation process is to understand the likely performance of a service change and how it might impact the business, the IT infrastructure, and other IT services. The process provides a consistent and standardized means of assessing this impact by assessing the actual performance of a change against its predicted performance. Risks and issues related to the change are identified and managed.

Objectives

The objectives of change evaluation include setting stakeholder expectations correctly and providing accurate information to change management to prevent changes with an adverse impact and changes that introduce risk being transitioned unchecked. Another objective is to evaluate the intended and, as much as possible, the unintended effects of a service change and provide good-quality outputs to enable change management to decide quickly whether or not a service change is to be authorized.

Scope

Effective change management means that every change must be authorized by a suitable change authority at various points in its lifecycle. Typical authorization points include before build and test starts, before being checked into the DML, and before deployment to the live environment. The decision on whether or not to authorize the next step is made based on the evaluation of the change resulting from this process. The evaluation report provides the change authority with advice and guidance. The process describes a formal evaluation suitable for use for significant changes; each organization will decide which changes need formal evaluation and which will be evaluated as part of change management.

Value to the Business

Change evaluation is concerned with value. Effective change evaluation will judge whether the resources used to deliver the benefit that results from the change represent good value. This information will encourage a focus on value in future service development and change management. CSI can benefit enormously from evaluation regarding possible areas for improvement within the change process itself and the predictions and measurement of service change performance.

Policies, Principles, and Basic Concepts

Let’s look at some of the key policies that apply to the change evaluation process. The first is that service designs or service changes will be evaluated before being transitioned. Second, although every change must be evaluated, the formal change evaluation process will be used only on significant changes; this requires, in turn, that criteria are defined to identify which changes are “significant.”

Change evaluation will identify risks and issues related to the new or changed service and to any other services or shared infrastructure. Deviation from predicted to actual performance will be managed by the customer accepting the change with the deviation, rejecting the change, or introducing a new change to correct the deviation. These three are the only outcomes of change evaluation allowed.

The principles behind change evaluation include committing to identifying and understanding the consequences of both the unintended and intended effects of a change, as far as possible. Other principles include ensuring that each service change will be fairly, consistently, openly, and, wherever possible, objectively evaluated and ensuring that an evaluation report is provided to change management to facilitate decision-making at each authorization point.

The change evaluation process uses the Plan-Do-Check-Act (PDCA) model to ensure consistency across all evaluations. Using this approach, each evaluation is planned and then carried out in multiple stages, the results of the evaluation are checked, and actions are taken to resolve any issues found.

Figure 26.1 shows the change evaluation process and its key inputs and outputs. You can see the inputs that trigger the evaluation activity and the interim and final evaluation reports that are outputs of the process. Where performance does not meet the requirement, change management is responsible for deciding what to do next.

Trigger

Let’s look first at the trigger for this process. The trigger for change evaluation is receipt of a request for evaluation from change management.

Inputs

Inputs to change evaluation are the service design package (which includes the service charter and service acceptance criteria), a change proposal, an RFC, a change record, and detailed change documentation. Other possible inputs are discussions with stakeholders and the test results and report.

Outputs

The outputs from change evaluation are interim and final evaluation report(s) for change management.

Interfaces

Change evaluation interfaces with a number of other processes. The main interfaces are as follows:

• Transition planning and support. Change evaluation works with transition planning and support to ensure that appropriate resources are available when needed and that each service transition is well managed.
• Change management. There is a critical interface with change management. The processes of change evaluation and change management must be tightly integrated, with clear agreement on which types of change will be subject to formal evaluation. The time required for this evaluation must be included when planning the change. In addition, it is change management that triggers change evaluation, and change management is dependent upon receiving the evaluation report in time for the CAB (or other change authority) to use it to assist in their decision-making.
• Service design coordination. This process provides the information about the service that change evaluation requires, in the form of a service design package.
• Service level management or business relationship management. Change evaluation may need to work with these processes to understand the impact of any issues that arise and to agree on the use of customer resources to perform the evaluation.
• Service validation and testing. This process provides change evaluation with information; the two processes must coordinate activities to ensure that required inputs are available in sufficient time.

Critical Success Factors and Key Performance Indicators

As with the other processes, the performance of change evaluation should be monitored and reported, and action should be taken to improve it. Here are two examples of CSFs for change evaluation and the related KPIs for each.

• Critical success factor: “Stakeholders have a good understanding of the expected performance of new and changed services.”
  • KPI: The reduction in incidents reported when a new or changed service fails to deliver the required utility or warranty
  • KPI: Increased customer satisfaction with new or changed services
• Critical success factor: “Change management has good-quality evaluations to help them make correct decisions.”
  • KPI: Increased percentage of evaluations delivered within the agreed timeframe
  • KPI: Reduced number of changes that fail or need to be backed out
  • KPI: Increased change management staff satisfaction when surveyed

Challenges

Challenges to change evaluation include developing standard performance measures and measurement methods across projects and suppliers, understanding the different stakeholder perspectives that underpin effective risk management for the change evaluation activities, and understanding (and being able to assess) the balance between managing risk and taking risks because this affects the overall strategy of the organization and service delivery. A further challenge is to measure and demonstrate less variation in predictions during and after transition.

The remaining challenges to change evaluation include taking a pragmatic and measured approach to risk and communicating the organization’s attitude toward risk and approach to risk management effectively during risk evaluation. Finally, change evaluation needs to meet the challenges of building a thorough understanding of risks that have impacted or may impact successful service transition of services and releases and encouraging a risk management culture where people share information.

Risks

The most common risks to the success of this process include a lack of clear criteria for when change evaluation should be used and unrealistic expectations of the time required. Another risk is that staff carrying out the change evaluation have insufficient experience or organizational authority to be able to influence change authorities. Finally, projects and suppliers who fail to deliver on the promised date cause delays in scheduling change evaluation activities.

Purpose

The purpose of the knowledge management process is to share perspectives, ideas, experience, and information; to ensure that these are available in the right place at the right time to enable informed decisions; and to improve efficiency by reducing the need to rediscover knowledge.

Objectives

The major objective of knowledge management is to insure that knowledge, information, and data is gathered, analyzed, stored, shared, used, and maintained throughout the service provider organization. The knowledge is managed through the service knowledge management system (SKMS), which provides controlled access to knowledge, information, and data appropriate for each audience. Having this available will have the following results:

• It will reduce the need to rediscover knowledge, so the service provider can be more efficient—able to deliver improved quality of service and achieve increased satisfaction while reducing cost.
• It will ensure that staff have a clear understanding of the value and benefits realized from the use of those services.
• It will improve management decision-making by providing reliable and secure knowledge, information, and data.

Scope

Knowledge management is relevant to all lifecycle sectors, not just service transition. The main scope of the process includes oversight of the management of knowledge and the information and data from which that knowledge derives. Excluded from the process is the capture, maintenance, and use of asset and configuration data, which, as you saw earlier, is the responsibility of the service asset and configuration management process.

Value to the Business

Knowledge management is important for all stages of the lifecycle, for all roles. It is particularly important for the transition of services because there is a strong focus on management of knowledge transfer as part of transition. This will enable the business to use the delivered service effectively.

Effective knowledge management delivers conformance with legal and other requirements, such as company policy and procedures. It provides documented requirements for retention and disposal of each category of data, information, and knowledge and ensures that data, information, and knowledge is current, complete, valid, and available to the people who need it when they need it. It also ensures that procedures are in place for the safe disposal of knowledge that is no longer required.

Policies, Principles, and Basic Concepts

Let’s now look at the policies, principles, and basic concepts that underpin knowledge management.

Data to Information to Knowledge to Wisdom

Knowledge management is typically displayed within the data, information, knowledge, wisdom (DIKW) model, as shown in Figure 26.2. You should remember this from your Foundation studies.

Let’s look at the model, and examine what is meant by each of these concepts:

• Data is a set of discrete facts. Most organizations capture significant amounts of data in highly structured databases such as service management and service asset and configuration management tools/systems and databases.
• Information provides context to data. An example of information is the average time to close priority 2 incidents. This information is created by combining data from the start time, end time, and priority of many incidents.
• Knowledge is composed of the tacit experiences, ideas, insights, values, and judgements of individuals. Knowledge puts information into an “ease of use” form, which can facilitate decision-making. An example of knowledge is that the average time to close priority 2 incidents has increased by about 10 percent since a new version of the service was released.
• Wisdom makes use of knowledge to create value through correct and well-informed decisions. Wisdom involves having the contextual awareness to provide strong common-sense judgement. An example of wisdom is recognizing that the increase in time to close priority 2 incidents is due to poor-quality documentation for the new version of the service.

The Service Knowledge Management System

The service knowledge management system (SKMS) holds all the knowledge relating to IT service management. This knowledge is underpinned by a large quantity of data gathered by various processes and held in the SKMS. One very important example is the configuration management system (CMS), which forms part of the SKMS. The CMS describes the attributes and relationships of configuration items, many of which are themselves knowledge, information, or data assets stored in the SKMS. We covered the CMS in the chapter about service asset and configuration management, Chapter 24, “Service Transition Processes: Service Asset and Configuration Management.”

Figure 26.3 shows a very simplified illustration of the relationship of the three levels, with configuration data being recorded within the CMDB and feeding through the CMS into the SKMS. The knowledge provided by the SKMS supports delivery of the services and informed decision-making.

The SKMS will contain many different types of data, information, and knowledge, including the service portfolio, the CMS, the DML, SLAs and OLAs and contracts, the information security policy, the supplier and contract management information system (SCMIS), budgets, and cost models. Many of these knowledge and information assets are configuration items. Changes to CIs must be under the control of the change management process, and details of their attributes and relationships will be documented in the CMS. Figure 26.4 shows examples of information that should be in an SKMS.

A knowledge management strategy is required. In the absence of an organizational knowledge management approach, action is required to establish the process within transition or IT service management. The strategy will address the governance model, roles and responsibilities, ongoing funding, policies, processes, procedures, methods and technology for knowledge management, performance measures, and how knowledge is to be identified, captured, and maintained.

Process Activities, Methods, and Techniques

An overall strategy for knowledge management is required. If the organization as a whole has no such strategy, it may be left to those responsible for IT service management as a whole or for service transition to impose knowledge management in their areas. The strategy should ensure that knowledge is shared with as wide a scope as practicable, covering direct IT staff, users, third-party support, and others likely to contribute to or make beneficial use of the knowledge.

If an organization has such a strategy, however, IT service management should be designed to fit within that overall organizational approach.

In either case, the strategy should address the following:

• The governance model, including any applicable legal or regulatory requirements
• Current and future roles and responsibilities
• Funding
• Policies, processes, procedures, and methods for knowledge management
• Technology and other resource requirements
• Performance measures

Knowledge needs to be transferred to other people and to other parts of the organization at specific points in the lifecycle. For example, knowledge transfer takes place to ensure that the service desk has optimum knowledge when a service is being transitioned into support. This would include information from release and deployment management such as known errors and diagnostic scripts from any of the technical support teams. Links with HR, facilities, and other supporting services need to be set up to facilitate the gathering and sharing of knowledge.

Knowledge rests on the management of the information and data that underpins it. To be efficient, this process requires an understanding of some key process inputs, such as how the data, information, and knowledge will be used, asking questions such as these:

• What knowledge is necessary?
• What do we need to monitor?
• What data is available?
• What is the cost of capturing and maintaining data?
• Does its value justify the cost?

We also need to consider applicable policies, legislation, standards, intellectual property rights, and copyright issues. Using the service knowledge management system reduces the costs of maintaining and managing the services.

Triggers

Now let’s look at the triggers, inputs and outputs, and process interfaces for knowledge management. Let’s look first at the triggers for this process. Knowledge management has many triggers relating to every requirement for storing, maintaining, and using knowledge, information, or data within the organization. For example, any of the following actions could trigger the need to store or retrieve knowledge:

• Business relationship management storing the minutes of a customer meeting
• Updates to the service catalog or service portfolio
• Modification of a service design package
• The creation of a new or updated capacity plan
• The receipt of an updated user manual from a supplier
• The creation of a customer report
• Updates to the CSI register

Inputs

Inputs to knowledge management include all knowledge, information, and data used by the service provider as well as relevant business data.

Outputs

The key output of knowledge management is the knowledge required to make decisions and to manage the IT services; this knowledge is maintained within an SKMS. The success of knowledge management depends in part on the participation of the relevant staff. For example, frontline operations staff will use the information about known errors to help when supporting users; they will also be responsible for gathering a lot of the data that will be stored in the SKMS, such as incident records. Problem management staff will be key users of incident data, while transition staff will capture data to be fed back to CSI and design.

Interfaces

Knowledge management has interfaces to every other service management process in every stage of the lifecycle. The SKMS can only be truly effective if all processes and activities use it to store and manage their information and data so that the maximum value can be extracted.

Critical Success Factors and Key Performance Indicators

As with the other processes, the performance of knowledge management should be monitored and reported, and action should be taken to improve it. Here are two examples of CSFs for knowledge management and the related KPIs for each.

• Critical success factor: “Availability of knowledge and information that helps to support management decision-making.”
  • KPI: Increased number of accesses to the SKMS by managers
  • KPI: Increased percentage of SKMS searches by managers that receive a rating of good
• Critical success factor: “Successful implementation and early life support operation of new and changed services with few knowledge-related errors.”
  • KPI: Reduction in the number of incidents and problems categorized as knowledge related
  • KPI: The increased percentage of successful service transitions

Challenges

Implementing knowledge management can be challenging. Challenges include justifying the effort needed to create a repository for existing knowledge, information, and data. There may be a perception that knowledge management is interfering in the work of other teams who manage their own information. The final challenge is persuading all the stakeholders of the benefits that a more cooperative and holistic approach to knowledge management can bring.

Risks

The risks to successful knowledge management are as follows:

• The emphasis could be on tools rather than on the creation of value.
• There could be insufficient understanding of what knowledge, information, and data are needed by the organization.
• There may be doubt as to the value of the process, leading to insufficient investment in the tools and people needed to support the SKMS.
• Without a clear knowledge management strategy, effort may be concentrated on knowledge capture with insufficient attention to knowledge transfer and reuse.
• The process may lose credibility if the knowledge and information that is stored and shared is out of date and irrelevant.
• If the benefits are not clear, stakeholders are unlikely to give the support and commitment the process requires.