When to Assess

Assessments can be carried out at any time, but it is good practice to associate an assessment to the improvement cycle (Plan-Do-Check-Act).

If we consider the Plan stage, there should be an assessment carried out as part of the project initiation. This is particularly important at the beginning of a process improvement initiative. When processes are being introduced, they should be assessed as part of the baseline for the improvement. Processes can be of many configurations and design, which increases the complexity of assessment data collection.

Planning can be a lengthy activity because incremental plans may be agreed on midstream during the project. Assessment taking place in the course of a process improvement—ensures that the project objectives are being met and can provide evidence that benefits are being achieved from the investment in time and resources.

When the process is in progress, we are part of the Do-Check stages. Assessment during a project—for example, at the conclusion of a project stage for process improvement—is important to validate the maturation of process and the process organization that was achieved through the efforts of the project team. Periodic reassessment following an improvement initiative will ensure that quality standards are maintained or further improvements are identified.

What to Assess and How

Setting the scope of the assessment is obviously a very important decision. A key consideration must be the objective of the assessment and what the expected future uses of the process assessments and assessment reports will be. Assessments can be targeted broadly at current processes or focussed on specific issues within the process environment.

There are three potential scope levels, namely process only; people, process, and technology; and a full assessment, including culture.

Advantages of Assessments

Assessments can provide an objective perspective of the current operational process state. This perspective can be compared to a standard maturity model and a process framework. Once a thorough assessment has been conducted, an accurate identification of any process gaps can be quickly completed, recommendations for remediation put forward, and action steps planned.

A well-planned and well-conducted assessment is a repeatable process. The assessment should be a useful management tool for measuring progress over time and establishing improvement targets or objectives.

Using an accepted industry-recognized maturity framework applied to a standard process framework allows an organization to compare its findings against a wider industry standard. This may be useful in promoting the organization for commercial tender.

An assessment provides information for the improvement cycle, answering the “Where are we now?” question and highlighting potential improvement areas.

Risks of Assessments

Of course there are risks to carrying out an assessment. It will only provide a snapshot of a specific state at a specific time. Dependent on the assessment mechanism used, you may be tying your organization into a particular vendor-specific choice of assessment and maturity framework. Occasionally, organizations find that the assessment and the achievement of the targets becomes an end in itself and the actual benefits and improvements that will help the organization are lost in the achievement of the maturity targets.

All assessments take resource effort, both for the practitioners of the processes and the assessors. It is important to understand the impact of this and realistically schedule the right amount of time. This is one of the many challenging aspects for assessments, but it is often overlooked.

Whatever the results, any assessment will require some interpretation and will therefore be subject to the experience, attitude, and approach of the assessor. It is important to try to establish objectivity as much as possible, but this may not be possible.

Consideration of who carries out the assessment is also important, as is ensuring that objectivity is retained through subsequent repeated audits. This is a particular challenge if the same assessor is used or the assessment is carried out internally. Knowing what was previously in place will not necessarily show where any new improvements are needed or if something has relapsed to an earlier maturity level.

Assessment Considerations

In the CSI journey, the decisions as to what to improve are critical to the overall results that can be achieved. Any discussion on improvements has to begin with the services being provided to the business. This could lead to improvements of the service itself or to process improvements that support the business service.

Figure 46.1 shows the relationships between services, processes, and systems.

Service improvements are governed by the improvement lifecycle. The improvement lifecycle is modeled after the Deming Cycle of Plan-Do-Check-Act. The cycle establishes a clear pattern for continual improvement efforts. Assessment will be an important input into the planning and part of the output from the planning stages.

Value of Processes vs. Maturity of Processes

For service management process improvement projects, one of the questions should address how mature we need our processes to be. The answer is tied directly back to the business. In other words, how important a process is to the business.

In Figure 46.2, you can see the value of a process mapped to the importance to the business using three examples—service level management (SLM), availability management (AM in the figure), and capacity management (CAP in the figure).

An assessment has shown that these processes are not very mature. This particular organization is changing its strategy for selling and delivering products and services to a web-based strategy. Because of the importance of capacity management and availability management to any organization that provides products and services over the Web, this company has to implement an improvement program for increasing the maturity of both processes, because without any improvement initiatives, this particular organization is putting itself at risk.

Having a low SLM process maturity will create some issues for CSI activities. SLM identifies new business requirements and provides information on what is currently being monitored and performance against targets. Without this information, CSI will have no baseline data for comparison.

The maturity of a process should ideally fall in the “safe” areas. If a process is immature but the business heavily depends on it, there is potentially a significant danger to the organization. If a process is very mature yet provides very little to the business, then an organization may be overinvesting resources and money. It is important to understand not only the value to the business, but also the relationship to other processes when making this assessment. Consider the impact of the removal of problem management to incident management, for example. A mature problem management process may be very proactive, and the benefit to the organization will be difficult to assess in business terms. But without it, the impact on incident management will definitely be adverse.

When CSI is looking at improving processes in support of IT services, it’s critical to understand the value of processes to a business as well as their function in the lifecycle as a whole.

Benchmarking Procedure

For benchmarking to be successful, it is important to identify your problem areas.

A range of research techniques may be required, such as informal conversations with customers, employees, suppliers, or focus groups to capture feedback. More formal approaches using marketing research and quantitative research can provide industry sector data. Internally, feedback can also be gained by using surveys and questionnaires. Often the result of a benchmarking activity will be process mapping and reengineering analysis. Quality control should be applied to variance reports to provide information on process achievements, and financial data will be used to understand the balance between cost and efficiency.

Benchmarking Costs

Benchmarking is a moderately expensive process, but most organizations find that it more than pays for itself. There are three main types of costs.

The costs associated with travel- and accommodation-related expenses for team members who need to travel to the site are known as visit costs. This is applicable for either internal or external assessors, but it is more likely when using an external organization because none of the team members will be based at the organization’s sites.

Time costs will be significant if the assessment is to be completed thoroughly. Members of the benchmarking team will be investing time in researching problems and finding exceptional companies to study and on visits and implementation. This will take them away from their regular tasks for part of each day, so additional staff might be required.

Once benchmarking is part of business-as-usual practice, it is important to capture and manage the data collected. It is useful to create and maintain a database of similar best practices and the companies associated with each best practice.

Value of Benchmarking

Benchmarking can be seen as valuable only if the results are clearly communicated. This should include displaying the gaps, identifying the risks of not closing the gaps, and assisting with the prioritization of development activities and facilitating communication of this information.

Benchmarks show profiles of existing quality in the marketplace and industry sector. Demonstration of quality by comparison to benchmarks can motivate staff and aid retention. Achievement of a quality standard can be a source of pride and self-confidence in employees because it shows that they work in an efficient environment.

Customers will be able to see that the organization is a good IT service management provider.

Optimizing service quality is key to all IT organizations to maximize performance and customer satisfaction and provide value for money. Using a benchmark as a comparison allows organizations to demonstrate their achievement.

Benchmarking as a Lever

It is common to hear staff say, “The way we do it is the best because this is the way we’ve always done it.”

Benchmarking is often a way to open an organization to new methods, ideas, and tools to improve their effectiveness. It can help break through resistance to change by demonstrating methods other than the ones currently employed and demonstrating evidence that others are using them successfully.

Benchmarking as a Steering Instrument

Benchmarking should be used as a management technique to improve performance. It is used to compare performance between different organizations or different units within a single organization undertaking similar processes.

It can be used as an ongoing method of measuring and improving products, services, and practices against the best that can be identified in any industry anywhere. It has been defined as “the search for industry best practices that lead to superior performance.” Benchmarking can support management in driving the direction of organizational change.

Benchmarking Categories

An internal benchmark is where an organization sets a baseline at a certain point in time for the same system or department and then measures to see how it is doing today compared with the baseline originally set. This type of benchmark is often overlooked by organizations (service targets are a form of benchmark), but they can be as useful as comparing to external benchmarks by showing an improvement progression.

Other benchmarking categories are comparisons with industry norms provided by external organizations, direct comparisons with similar organizations, and comparison with other systems or departments within the same company.

Benefits

Using benchmark results should help deliver major benefits in achieving lower prices and higher productivity on the part of the service provider. This should include identifying efficiencies by comparing the costs of providing IT services and the contribution these services make to the business with what is achieved in other organizations. This helps the organization to identify areas for improvement.

Benchmarking will also demonstrate effectiveness in terms of actual business objectives realized compared with what was planned. To obtain the maximum benefit, it is necessary to look at economy, efficiency, and effectiveness rather than focusing on one to the exclusion of the others.

Who Is Involved?

Within an organization, there will be three parties involved in benchmarking. Each has a different perspective on the results of benchmarking and how to apply them.

• The customer or the business manager responsible for acquiring IT services to meet business objectives. The customer’s interest in benchmarking would be, “How can I improve my performance in procuring services and managing service providers, and in supporting the business through IT services?”
• The user or consumer, namely anyone who uses IT services to support their work. The user’s interest in benchmarking would be, “How can I improve my performance by utilizing IT?”
• The internal service provider who provides IT services to users under service level agreements negotiated with and managed by the customer. The provider’s interest in benchmarking would be, “How can we improve our performance in the delivery of IT services that meet the requirements of our customers and are cost-effective and timely?”

There will also be participation from external parties:

• External service providers provide IT services to users under contracts and service level agreements negotiated with and managed by the customer.
• Members of the public are increasingly becoming direct users of IT services, and this is challenging when attempting to benchmark against their needs.
• It is important not to forget the input that will be required from benchmarking partners, that is, other organizations with whom comparisons are made in order to identify the best practices to be adopted for improvements.

What to Benchmark

Differences in benchmarks between organizations are normal. Each organization will have a slightly different setup; no two will be exactly the same. Direct comparison with similar organizations is most effective if there is a sufficiently large group of organizations with similar characteristics.

Benchmarking techniques can be applied at various levels, from relatively straightforward in-house comparisons to an industry-wide search for best practice. Benchmarking should follow the continual service improvement seven-step process to ensure that appropriate data is collected, analyzed, presented, and acted on.

Comparison with Industry Norms

ITIL is itself an industry-recognized best practice. The core publications provide documented guidance on benchmarking and process assessment. There are many organizations that provide IT service management consultancy and professional expertise in benchmarking, which may be useful to an organization. Use of maturity models is supported by a number of frameworks, and there is Capability Maturity Model Integration (CMMI), which is widely recognized.

Total cost of ownership (TCO), developed by Gartner, has become a key measurement of the effectiveness and efficiency of services. TCO is defined as all the costs involved in the design, introduction, operation, and improvement of services within an organization from its inception until retirement. TCO is often used to benchmark specific services in IT against other organizations, for example, managed service providers.

Benchmark Approach

Benchmarking will establish the extent of an organization’s existing maturity with best practice and help in understanding how that organization compares with industry norms. Deciding what the key performance indicators (KPIs) are going to be and then measuring against them will give solid management information for future improvement and targets.

There are two basic approaches: either an internal benchmark, which is completed internally using resources from within the organization to assess the maturity of the service management processes against a reference framework, or an external benchmark, completed by an external third-party company. A third party will probably have its own proprietary models for the assessment of service management process maturity.

Viewed from a business perspective, benchmark measurements can help the organization to assess IT services, performance and spend against peer or competitor organizations, and best practice, both across the whole of IT and by appropriate business areas. There are a number of questions often asked about IT, such as: How does IT spending compare to other similar organizations—overall, as a percentage of revenue, or per employee? It is hard for an organization to understand whether or not it is spending too much compared to similar organizations for basic functions such as payroll or to compare spending across business units, locations, or processes. Competitors are unlikely to divulge details about spending on these types of services because it will be perceived as commercially confidential information. So a benchmarking activity against an agreed norm is often the only approach to support business understanding and justification of the costs of IT.

Benchmarking activities need to be aligned to the business. If carried out thoroughly, a benchmarking exercise, whether completed internally or externally, will incur significant costs. It is important that the benchmark is targeted to identify areas that will be of most value to the business.

The approaches to benchmarking can include an assessment of the cost and performance for internal service providers or the price and performance for external service providers. Or it can focus on the performance of processes against industry best practice. The comparison to industry sector or peer information relating to financial performance of IT is another assessment, as is effectiveness based on customer satisfaction ratings and business alignment.

Whichever approach is adopted, it is important that the context for benchmarking requires information about the organization’s profile and complexity and relative comparators. An effective and meaningful profile contains four key components.

The company profile provides basic information. Company size, industry type, geographic location, and types of user are typical of data gathered to establish this profile.

There also needs to be an understanding of the current assets because the IT assets within the organization may include operational IT, desktop and mobile clients, peripherals, and network and server assets.

It is important to understand current best practices, including the policies, procedures, and/or tools that improve returns and their maturity and degree of usage. Understanding the organization includes information about the end-user community, the types and quantities of varied technologies in use, and how IT is managed.

There are a variety of IT benchmarking types available separately or in combination:

• Cost and performance for internal service providers
• Price and performance for external service providers
• Process performance against industry best practice
• Financial performance of high-level IT costs against industry or peers
• Effectiveness benchmarking, which considers satisfaction ratings and business alignment at all levels

Benchmarking will establish the extent of an organization’s existing maturity with best practice and will help in understanding how that organization compares with industry norms.

Design and Develop a Service Measurement Framework

It is always challenging for an organization to create a measurement framework that supports the business requirements. One of the key factors for this is the definition of what success looks like. We need to be mindful of both the past and the future; measurement should allow for the identification of future improvement as well as report on past performance.

Whether measuring one or multiple services, the following are key to a successful service measurement framework.

• The origins of the framework and defining what success looks like; in other words, what are we trying to achieve and how will we know when we’ve achieved it?
• Ensuring that we are building the framework and choosing measures that will provide us with information to make strategic, tactical, and/or operational decisions.

It is important to select measures that will deliver the data and information we need based on agreed targets within IT and the business.

There are some critical elements that should be included in a service measurement framework. For example, the framework should be integrated into business planning and focused on business and IT goals and objectives. It should support cost-effectiveness, with a balanced approach to the measures applied that can be sustained over a period of time and withstand change. The framework must clearly identify the performance measures that will encourage the behaviors desired and be accurate, timely, and reliable. It is also important to ensure that the roles and responsibilities are clearly defined, so there is no doubt about who defines the measures and targets, who monitors and measures, and who gathers and analyzes the data and prepares the reports.

Different Levels of Measurement and Reporting

A service management framework should be built on different metrics and measurements so that the end result is a combined view of the way the individual components support the overall service. This in turn should provide information to the key performance indicators, allowing us to ensure that targets are being achieved. This will then be the basis for creating a service scorecard and dashboard.

The service scorecard can then be used to populate an IT scorecard or overall balanced scorecard. Figure 46.4 shows a diagrammatic representation of the multiple levels that need to be considered when developing a service measurement framework.

Service Management Process Measurement

There are four major levels to report on. They are shown in Figure 46.5.

The bottom level contains the activity metrics for a process, and these are often volume-type metrics such as number of requests for change (RFCs) submitted, number of RFCs accepted into the process, number of RFCs by type, number approved, number successfully implemented, and so on.

The next level contains the KPIs associated with each process. The activity metrics should feed into and support the KPIs. In turn, the KPIs will support the next level, which is the high-level goal such as improving service quality, reducing IT costs, or improving customer satisfaction.

Finally, this high-level goal will feed into the organization’s balanced scorecard or IT scorecard. When first starting out, it is important not to pick too many KPIs to support the high-level goal(s). Additional KPIs can always be added at a later time.

Creating a Measurement Framework Grid

As a significant part of this approach, best practice recommends that the organization create a framework grid to set out the high-level goals and define which KPIs will support the goal, and also which category the KPI addresses.

An example of this can be seen in Table 46.2, which is an extract from the CSI core publication.

In this example, the high-level goal relates to availability and reliability of a service, with a qualitative KPI to demonstrate a percentage improvement in overall availability. We can see the components measured and the desired target achievement as well as those responsible for the measurement.

When considering performance, it is important to recognize that there are different elements that combine to give an overall perception of achievement. These can be classified as compliance, quality, performance, and value.

• Compliance is a measure to demonstrate if we are doing something.
• Quality allows us to measure how well are we doing something.
• Performance demonstrates the speed and urgency of carrying out something; in other words, how fast or slow are we doing it.
• And last but by no means least, we have value, and this determines if what we are doing is making a difference.

How Many CSFs and KPIs?

It’s a valid question—how many critical success factors and KPIs should we have?—and opinions on this are varied. The more KPIs there are, the more complex the reporting model and analysis required to interpret them will be.

Good practice suggests that CSFs should be supported by a number of associated KPIs, but there is no defined number for either. Even a very mature organization is unlikely to have more than five CSFs per process, with no more than five KPIs per CSF. But that still adds up to a potentially high number of metrics. So it is recommended that in the early stages of a CSI program, only two or three KPIs for each CSF are defined, monitored, and reported on. As the maturity of a service and service management process increases, further KPIs can be added.

Remember, KPIs will change over a period of time as their importance to the business and the maturity of the service provision alters, and this may have effect on other KPIs and processes. But the changes to KPIs must be carefully considered so that trending information or the value of the metric is not lost by too frequent alterations.

Tension Metrics

Tension metrics ensure that the team efforts stay in balance by measuring a combination of the elements that deliver a successful support team. There should be a balance of resources (the people and the money), the features (the product or service and the quality of that product or service), and the schedule (an element of timeliness).

Focusing on one factor above others will cause an imbalanced approach. For example, if too much focus is placed on delivering to a schedule, quality may be impacted by the increased speed to delivery. Similarly, concentrating too much on service quality may impact on the financial resources in use because high quality is costly and the end product needs to be cost justifiable.

Tension metrics are designed to measure a balanced approach so that if one measure drives a specific behavior, such as a service desk answering calls quickly, we have a measure in place to ensure that quality does not suffer to beat the time constraint. This is why there should always be a suite of measures in place; concentrating on any one aspect may harm the overall service.

Tension metrics should not cause conflict with goals or objectives but should ensure that the team maintains an overall focus on quality of service across all aspects of delivery.

Goals and Metrics

Goals and metrics are important for all stages of the lifecycle—from strategy, where the organization will decide on how IT will be funded; through design, where business requirements are translated into IT solutions; through transition, where they are made a reality; to operation, where the business finally sees the direct value of IT. Throughout the lifecycle, all stages must keep the business goals and requirements in mind, and these should be reflected in the goals and objectives for each stage.

Interpreting and Using Metrics

Results must be interpreted in the context of the objectives for the measures as well as any environmental or external factors. If results are considered out of context, they may be misinterpreted if extenuating factors have affected them. It is important to review the measures to ensure that the chosen indicators have worked and that the results are contributing to the overall objective of the service or process.

To make sure reports are useful and meaningful, it is important to ensure that the generated results are making sense. If the output does not show a probable or viable outcome, then investigation must take place into how this could have occurred. The investigation is not designed to identify blame but to rectify an error in reporting so the required results can be produced.

The following questions need to be answered to ensure that the results are verified properly:

• How did we collect this data?
• Who collected the data?
• What tools were used to collect the data?
• Who processed the data?
• How was the data processed?
• What could have led to the incorrect information?

Before starting to interpret results, you should always ensure that you have sufficient information about the data elements that have been used and the purpose of the results. It is important to understand the expected normal range for the results so that it is possible to identify any anomalous results or exceptions.

It is easy to jump to conclusions incorrectly; for example, a downward trend in calls opened at the service desk may be caused by a wide range of scenarios. There could have been a change in the way support is offered, perhaps by the introduction of self-service, or there could have been a failure in the telephone system. It should be noted if there have been any changes that might have triggered a set of results, and where insufficient information is available from simply reviewing the results, make sure the appropriate people are included in a discussion.

Using Measurement and Metrics

Metrics can be used for multiple purposes, and it is important to understand the objectives you are trying to achieve so that you select the correct purpose. For example, metrics can be used to validate a decision, such as whether you are supporting the strategy and vision of the business. They may also be used for justification, to answer the question, Do we have the right targets and metrics? Of course, we all understand that metrics can drive behaviors, so they can be used to direct and change people’s actions based on factual data. Often metrics are used to identify when an intervention needs to take place or to take corrective actions such as identifying improvement opportunities.

As always, it is important to ensure that there is a balanced approach; focusing only on identification of improvement can have a negative impact on staff morale.

Service measurements and metrics should be used to drive decisions. Depending on what is being measured, the decision could be strategic, tactical, or operational.

CSI will have to manage many improvement opportunities, but often with only a limited budget to address them, so decisions must be made. Which improvement opportunities will support the business strategy and goals, and which will support the IT goals and objectives? What are the desired return on investment (ROI) and value on investment (VOI) opportunities? Note that the measures and metrics are always being reviewed against desired business goals and outcomes to ensure that IT continues to align with and meet business expectations.

Measures by themselves may tell the organization very little unless there is a standard or baseline against which to assess the data. Measuring only one particular characteristic of performance in isolation is meaningless unless it is compared with something else that is relevant. Measures of quality allow for measuring trends and the rate of change over a period of time. The following comparisons may be useful:

• Comparison of the assessment against the baseline or agreed standard. It is important to understand the criteria for any deviation from the standard so that you make only necessary improvements instead of responding to every discrepancy.
• Comparison against a target or goal in an SLA is important so there is a clear understanding of fluctuations in service quality. This will strengthen a relationship between service provider and customer by demonstrating engagement and forward planning.
• Comparison with other organizations is also helpful, but it is necessary to ensure that the strategy, goals, and objectives of other organizations align with yours.
• Comparison over time, such as day to day, week to week, month to month, quarter to quarter, or year to year, is a commonly used approach for trend analysis. Remember to ensure that you are still comparing relevant and appropriate data samples. If a measure has been altered, it may be that the comparison is no longer valid.
• Comparison between different business units and services allows the organization to ensure consistency across the enterprise as a whole.

Using measures and metrics is a powerful mechanism for the identification of improvements, and trend analysis can provide information to predict future performance.

It is also important to ensure that your measures take into consideration external factors, such as political influences or market forces outside of your organization.

Individual metrics and measures by themselves do not communicate very much from a strategic or tactical point of view. Some types of metrics and measures are often more activity based than volume based, but they have value from an operational perspective. They might include the services used and which customers are using those services. Understanding service usage both for the time of day and how often will help you understand the demand requirements, as will the medium through which it is accessed (for example, whether it’s internal, external, or web based). At a lower level of granularity, the performance and availability of components also provides useful information on the quality of the service delivered.

Each of these measures by themselves will provide some information that is important to IT staff (particularly the technical management staff), but it is the examination of all the measurements and metrics together that delivers the real value.

It is important for someone to take responsibility for looking at these measurements as a whole and to analyze trends and interpret the meaning of the metrics and measures.

Creating Scorecards That Align to Strategies

Reports and scorecards should be linked to overall strategy and goals. Using a balanced scorecard approach is one way to manage this alignment. In Figure 46.7, there is an illustration of how the overall goals and objectives can be used to derive the measurements and metrics required to support the overall goals and objectives.

The arrows point both ways because the strategy, goals, and objectives will drive the identification of required KPIs and measurements, but it is also important to remember that the measures are input into KPIs and the KPIs support the goals in the balanced scorecard.

It is important to select the right measures and targets to be able to answer the question of whether the goals are being achieved and the overall strategy supported.

Creating Reports

When creating reports, it is important to know their purpose and the details required as well as the time frame to which they relate. Reports can be used to provide information for a single month, or a comparison of the current month with other months to provide a trend for a certain time period. Reports can be used to show whether service levels are being met or breached.

Before starting the design of any report, it is important to understand some key facts, which should be incorporated into the design. One of the first items to consider is the target audience. Most senior managers don’t want a report that is 50 pages long. They like to have a short summary report and access to supporting details if they are interested.

Also important is understanding what the report will be used for because this will make a difference to the content and the way that it is presented and interpreted. Basic information about roles and responsibilities for the creation and production of the report and the frequency of reporting will also be part of the design. The audience will drive the information that is to be shared or exchanged. For example, if the report is going to be read by senior managers, then it should be short, informative, and usable without sacrificing readability.

The report format must meet the needs of the audience but be repeatable and aid the understanding of the data.

The majority of service management tools provide out-of-the-box functionality for reporting, including some standard reports providing basic content. One of the criteria for a good service management toolset is that it has not only this capability, but also the capability to produce customized reports to meet the individual customer’s requirements. This should include the ability to generate output in an acceptable media and format, such as web-based, automatically generated reports.

Usually reports are set up to show the results for a service, with supporting reports giving individual measurements on components and the health of a service management process by using process KPI results and functional reports, for example, telephony reports for the service desk.

Table 46.3 includes some examples of key performance indicators, but it is important to apply those that match your business and organizational requirements and strategy.

There are a wide variety of techniques used to measure IT and IT service effectiveness and efficiency, and they are often combined. CSI should be responsible for measurement of the quality of service (and corrections if the quality of service is below targets) while ensuring that IT is still operating within the financial constraints of the organization. CSI will measure progress, understand achievement against targets, and make corrections to improvements where required to remain on track.

It is tempting to allow reporting and measurement to become an end in itself because often the effort and analysis required for report generation requires a separate team of people within the department. CSI should ensure that the goal of reporting remains clearly focused on the progress toward the achievement of business goals and objectives and not simply chasing targets and producing statistics.

Cascading the Balanced Scorecard

Many organizations are structured around strategic business units (SBUs). Each business unit will focus on a specific group of products or services offered by the business. Once a balanced scorecard has been defined at the SBU level, it can be cascaded down through the organization.

Many organizations use scorecards in all departments, even at the board level, because for each strategic business-level measure and related target, business units or departments can define additional measures and targets that support the strategic goal and target. Action plans and resource allocation decisions can be made with reference to how they contribute to the strategic balanced scorecard.

The Balanced Scorecard and Measurement-Based Management

The Balanced Scorecard approach covers a number of important aspects, including customer-defined quality of service, continual improvement, employee empowerment, and measurement-based management and feedback.

The balanced scorecard is complementary to total quality management (TQM) and uses the same approach of feedback for internal business process outputs, but has an additional feedback loop for the outcomes of business strategies. This ensures a more complete approach to overall quality across the organization. To achieve this, metrics should be developed based on the priorities of the strategic plan. It is this plan that provides the key business drivers and criteria for metrics that managers most desire to watch. Services and processes can then be designed to collect information relevant to these metrics. Remember, metrics and measurements are part of the design of a service, as covered in the service design lifecycle stage.

Metrics are valuable because they provide a factual basis for defining feedback from some key areas. Strategic feedback shows the present status of the organization; information is gathered from many perspectives for decision-makers. Improvement requires input from diagnostics on a continuous basis, and performance trends can be tracked over time. It is always important to make sure the measures themselves are under continuous review because business requirements change and metrics should change to reflect this. Metrics should also be used to support forecasting methods, providing a quantitative input to the approach.

Purpose

SWOT analysis is a strategic planning tool. It is used to evaluate the strengths, weaknesses, opportunities, and threats associated with a project, business venture, or any other situation that requires decision-making.

How to Use SWOT Analysis

When SWOT analysis is used, the most important factor is to define the desired end state or objective. All the participants in the process must agree, and it must be clear and explicit. Without this clarity, the analysis will not be effective because each element is using the objective to define the specifics for each step. Because the subsequent actions following the analysis will be driven by the results, it is important to ensure that the objective is “SMART” (specific, measurable, achievable, relevant, and time-bound).

If we consider each step in turn, the analysis becomes clear as we define the action taking place. Strengths are the internal attributes that will be helpful in achieving the agreed objective, whereas weaknesses are internal attributes that will be harmful. The analysis then looks at external factors. Opportunities are those factors that will be helpful in achieving the agreed objective, and threats are external factors that will be harmful.

An accurate SWOT analysis is a useful planning tool, helping to answer questions such as, How can we use the strengths to our advantage? and How can we stop the weaknesses? Having a clear understanding of external factors is important; even if we are unable to change them, we may be able to adapt our practices to exploit them or mitigate against their impact.

Scope, Reach, and Range

A SWOT analysis can be performed at various levels throughout the organization. It can take place at an individual, departmental, divisional, or even corporate level. It is important to consolidate the results of the analyses, and this should be from the bottom up so that the lower levels of the hierarchy can be completed before the next is attempted.

For example, if the individual members of a functional team each perform a SWOT analysis to capture their individual perspective, the next SWOT analysis should be based on the team. Then multiple teams can perform the analysis until eventually the departmental level is reached. This would continue up to the corporate level.

It is also possible to conduct a SWOT analysis for a service or a process.

Common Pitfalls of a SWOT Analysis

It is important to align the SWOT analysis with the business vision, mission, goals, and objectives. If the end state of the analysis is not properly identified at the start, it may result in wasted resources and potentially failure. There are a number of common errors that can take place when carrying out a SWOT analysis.

One of the most common is conducting a SWOT analysis before defining and agreeing on the end state. Another common error is to confuse the external opportunities with the strengths in the internal organization. It is important to keep them separate. A further mistake is to confuse opportunities with possible strategies. SWOT is a description of conditions, while possible strategies define actions.

Establishing a Business Case

A business case needs to identify the reason for undertaking a service or a process initiative, including the specification of the data and evidence that needs to be provided to prove the costs and expected benefits.

It is important to remember that process redesign activities are complex and may be more costly than assumed. The same can be said for the impact of organizational change, and with the introduction of organizational and process change comes the potential requirement to improve competencies and tools, adding further cost to the improvement.

It is important not to limit the business case to return on investment but to include the value that a service improvement will bring to the organization. Value on investment can be measured as the improvement is implemented, whereas return on investment can really only be demonstrated once the implementation of the improvement has concluded. Working collaboratively with the business, it should be possible to identify the value the implementation brings to the business. Examples of business value measures include the time to market, customer retention, and the increase in organizational market share.

IT can demonstrate its contribution through gains in agility, managing and enhancing knowledge, and a reduction in costs and risk. IT should begin by defining the types of business values that each improvement will contribute.

Business Cases in a Data-Poor Environment

It is often the case that an organization that intends to carry out service improvements is doing so in a situation where the lack of process means that there is no evidence to prove the expected benefits, value on investment, or return on investment.

There is an approach that circumvents this situation by gaining approval to establish basic measurement capabilities as a means of gathering consistent data for future analysis. This may be as simple as ensuring that all IT staff record data in a consistent fashion or start measuring activities or outcomes that are not currently captured. After an agreed period of data capture, some evidence will exist to support (or perhaps not support) a process improvement initiative.

Another approach is to undertake a process maturity assessment of current processes, but this activity will identify only the absence of process and/or data. A process maturity assessment will not in itself provide the data to justify how much to spend on improving process. So often both approaches are used so that consistency is achieved with an understanding of how well processes are being followed, and from this, measures of value can be established.

It is important that once the decision to start capturing and reporting on data is made, an initial baseline is created so improvements can be measured against it.

Measuring Benefits Achieved

In the business case, we can identify estimated benefits, but eventually we need to measure achievements. These measurements show whether the improvement activity achieved the intended outcomes and should also consider whether the envisaged improvements were realized by measuring the benefits arising from the improvements. It is also important to demonstrate that the target ROI and the intended value-add was actually achieved (VOI).

Continual service improvement is cyclic, and the outcomes of measurements will lead to further process improvement actions being reevaluated.

It is important to ensure that enough time has passed before measuring the benefits. Some benefits will not be immediately apparent, and it is likely that benefits will continue to change over time as ongoing costs and ongoing benefits continue to change.

A further consideration in the measurement of benefits is that data quality and measurement precision pre- and postimprovement could be different. This may invalidate direct comparison, so there may be a requirement for the data to be normalized before validating benefits.

Reporting Policy and Rules

It is important to agree with the business on the policies and rules for the reports. Gain agreement in advance regarding the potential target audience, and then agree on what information the business requires about the service. During the design stage of the lifecycle, agreement should be sought on what is going to be measured and reported, including definitions of terms and boundaries. It is important to provide clarity on the mathematics used for all the calculations that are used. The scheduling and delivery mechanism for the reports and who will have access to them should be stipulated, as well as the attendees for the review and discussion meetings. It is good practice to agree on these elements in the design stage, but we should also remain flexible for alterations in the operations stage of the lifecycle.

Right Content for the Right Audience

It is important to ensure that the right content is provided for the right audience. It is good practice to apply the right policies for each target group because the needs of one customer may not be the same as another.

Once the framework, policies, and rules are in place, automating suitably styled reports is a task of translating flat historical data into meaningful business views. These will need to be annotated around the key questions, threats, mitigations, and improvements that have been identified in the report. Reports can then be presented via the medium of choice—for example, paper-based hard copies, online soft copies, web-enabled dynamic HTML, current snapshot whiteboards, and real-time portal/dashboards.

Simple and effective customizable and automated reporting is vital for an ongoing reporting schedule, which provides satisfactory reporting to the business.

It is also important to recognize that the initial schedule and content for reports may change over time as the business needs change. The end result should be targeted reporting that is clear, unambiguous, and relevant, delivered to the correct recipient in a medium and manner that promotes accessibility and use.

Availability Management

Availability management’s methods are part of the measuring process—gathering, processing, and analyzing activities. When the information is provided to CSI in the form of a report or a presentation, it becomes part of CSI’s gathering activity.

When used by availability management, this activity provides IT with the business and user perspective about how failures and faults in the infrastructure and underpinning process and procedures impact the business operation. The use of business-driven metrics can demonstrate this impact in real terms and help quantify the benefits of improvement opportunities.

The Expanded Incident Lifecycle

The expanded incident lifecycle, as shown in Figure 46.9, provides a technique to help with the technical analysis of incidents affecting the availability of components and IT services.

It is made up of two parts: time to restore service (also known as downtime) and time between failures (also known as uptime). There is a diagnosis part to the incident lifecycle as well as repair, restoration, and recovery of the service.

Using the other techniques in availability management, it is possible to review each element of the incident management process and apply to this review a continual service improvement activity to address issues in incident management. Management of infrastructure is often reliant on the information from the analysis of mean time between failures, mean time between system incidents, and the mean time to restore service.

Capacity Management

The capacity management process must be responsive to changing business requirements for processing capacity. New services are required, and existing services will require modification to provide extra functionality. Old services will become obsolete, freeing up capacity. Capacity management must ensure that sufficient hardware, software, and personnel resources are in place to support existing and future business capacity and performance requirements.

Similar to the availability management process, the capacity management process should play an important role in helping the IT support organization recognize where it can add value by exploiting its technical skills and competencies in a capacity context. Capacity management should use the continual improvement technique and apply this to technical capability. It is possible to do this either in small groups of technical staff or in a wider group within a workshop environment.

The information generated by the capacity management process should be made available to CSI through the capacity management information system (CMIS), a database that should form part of the service knowledge management system.

As you will remember from your Foundation course, capacity management has three subprocesses: business, service, and component capacity management.

Component Capacity Management

The component capacity management subprocess’s prime objective is to identify and understand the capacity and utilization of each of the components of the IT infrastructure. This is where the technical management expertise will be utilized. This ensures the optimum use of the current hardware and software resources in order to achieve and maintain the agreed service levels.

As in service capacity management, the key to successful component capacity management is to preempt difficulties wherever possible.

It is important to understand how the three subprocesses tie together. Let’s look at the example in Figure 46.10 and the requirements in Table 46.4

	Marketing	Sales	Finance
Employees	15	40	5
Number of emails per day	100	200	50
Size of attachment	10 Mb	5 Mb	10 Mb
Frequency of large attachment	Infrequent	Very frequent (contracts)	Often
Requires remote access	No	Yes	Yes
Requires handheld computer	No	Yes	No

Copyright © AXELOS Limited 2010. All rights reserved. Material is reproduced under license from AXELOS.

There are three services (A, B, and C) and three departments (Marketing, Sales, and Finance). Service A is used by all three departments. Service B is used only by Marketing and Sales. Service C is used only by Finance.

Each of the subprocesses will have a part to play in understanding the management of capacity in the organization. Changes in business focus should be communicated through business capacity management, but capacity planning is often carried out a year in advance, and it is very difficult to be accurate this far ahead, so it is important for business capacity management to be constantly informed of any changes by the organization. So if marketing provides business plans for an increase in sales, then this will have an impact on all three departments. Any of the services used will potentially be impacted. Marketing services will be required to take on greater capacity, while sales services will have a higher throughput, and so will the finance services as more sales go through.

It is also important for the activities in service capacity management to keep up with changes in service level requirements in real time, and these activities can be used to support the forecasting in business capacity management for the marketing department’s predicted increase in sales.

Component capacity management is highly technical, but the reports and information it produces are used by service capacity management to continue to monitor the capacity capability of the service as a whole. Each of the subprocesses is important in managing the organizational requirements.

IT Service Continuity Management

IT service continuity management (ITSCM) allows an organization to manage IT risks through a process of identifying what is important to the stakeholder, mitigating against the risks the organization chooses to take, and ensuring that the business processes will continue to operate throughout a disruption. CSI reviews the requirements of the organization, and improvements should be managed through the change process to ensure that they are reflected in the continuity plans.

Business continuity management is concerned with the management of risks at an organizational level, which are then supported by the ITSCM plans. The BCM process involves reducing the risk to an acceptable level and planning for the recovery of business processes should a risk materialize and a disruption to the business occur.

Problem Management

CSI and problem management are closely related because one of the goals of problem management is to identify and permanently remove errors that impact services from the infrastructure. This directly supports CSI activities of identifying and implementing service improvements.

Problem management also supports improvement activities through trend analysis and the targeting of preventive action. Although problem management activities are generally conducted within the scope of service operation, CSI takes an active role in the proactive aspects of problem management because it is here that the process is used to identify and recommend changes that will result in service improvements.

Knowledge Management

Knowledge management is a key support for CSI because capturing, organizing, assessing for quality, and using knowledge plays a large part in CSI activities. An organization has to gather knowledge and analyze what the results are in order to look for trends in service level achievements and the results and output of service management processes. This knowledge may be used to identify improvement opportunities for inclusion in the CSI register, which will then be reviewed and prioritized. It will also be used for contributing to service improvement plans and initiatives.

Knowledge management is constantly changing, in line with the technological advances in IT. The rate of change in the IT industry has opened up new opportunities for knowledge sharing and collaboration, not least of which is the extensive use of the Internet by corporations and end users. Staff turnover requires that corporate knowledge is captured centrally rather than being dependent on the individual because it is more common for individuals to change companies throughout their career.