Chapter 8
Assessment and Prioritization of Business Analytics Projects
The question of how to prioritize business analytics (BA) projects leads to two more questions:
- In which order should the BA initiatives be implemented?
- Which initiatives should not be implemented at all?
In the radio station case study in Chapter 1, we used one simple financial rule of thumb as our business case. That rule decided that the project should be implemented. Assessing BA initiatives in the real world, however, is somewhat more complicated. To answer the above two questions, we'll use a business case. A business case is an analytical tool that can provide support to decisions about whether to implement a BA project.
In the last section of this chapter, however, we will also look into how BA can be implemented in a wider perspective. So in this section, we will briefly examine how to make a full strategic roadmap, which can make an organizations journey into the analytical age controlled and predictable. This also means that instead of narrowing it down to a per initiative view we will lift it up to a per business process perspective. After reading this section, more details on how to make strategic analytical roadmaps are available on BA‐support.com in a separate paper, or the authors may be contacted with more questions.
IS IT A STRATEGIC PROJECT OR NOT?
When prioritizing projects, it's important to decide whether a given project is strategic. If it is, we don't have to assess whether the project should be carried out on the basis of a business case. Rather, we must expect this assessment to have been undertaken already from the strategic side. We do, however, have to ensure that sufficient means have been set aside for the project and, if this is not the case, we must decide which budgets are to cover the costs of the project.
However, if the project is not specified as part of the company's strategy, it means that it is requested based on the expectation that it will render improved business performance. This is called a bottom up–driven initiative because it comes from the operational environment. The opposite is called a top down–driven initiative, which is activated from the strategy.
Typically, projects that are not initiated from the strategy are prioritized in relation to other projects based on a business case approach. A business case performs the simple math of relating costs to the financial gains of a project; in this way, we can assess from a purely financial point of view whether we get the best return on investment. In Exhibit 8.1, we've made a small model showing how projects may be compared. This model is naturally not exhaustive, but can assist in creating an overview of different project candidates and their different natures.
Exhibit 8.1 BA Project Costs Compared with Benefits
If we have a project with low costs and a high return, this project will obviously be preferred to a project with the same costs, but lower return. Likewise, projects with high costs and low return would be rejected. A final possibility is a project with high value creation and high resource consumption. In this case, an assessment should be made as to whether the project can be lifted into a strategic context and, if not, whether it is then still relevant. It goes without saying that a project like that will take up a lot of resources in the departments that will be responsible for its implementation.
In other words, even if there is a positive business case for the project, it may be necessary to dismiss it due to its demand on resources and the fact that the project process will adversely affect the company's agility. By agility, we mean the ability to make fast decisions and to respond quickly to new opportunities, which could be golden opportunities, resulting from the implementation period of this major project. The large project therefore entails certain opportunity costs for the organization during its implementation. That is, we must weigh the added value of any golden opportunities that must be disregarded, as well as the opportunity to quickly start up strategic initiatives that may turn up along the way.
As illustrated in Exhibit 8.1, we can build a business case on the weighting of financial advantages and disadvantages of the business case. In Exhibit 8.2, we break this down even further, as an introduction to the following sections, which are about costs and advantages, respectively.
Exhibit 8.2 Return on Investment (ROI)
First, Exhibit 8.2 points out that costs related to the implementation of information technology (IT) solutions are rarely one‐off costs; we're typically looking at some additional future costs. Second, the exhibit shows that we must separate and arrange these according to increased value for the users of the system and according to savings, when we look at the advantages created by a given project.
Uncovering the Value Creation of the Project
The difficult thing about making business cases for BA projects is that they do not create value in themselves. Only when the subsequent improved decision making is experienced is value creation realized at an organizational level. It may therefore be tempting to list IT and implementation costs on the one hand and then a number of advantages on the other. However, the truth is often more complicated than that, because we usually move from having one business process to having another. This means that to be able to make a real estimate, we must look at how much more expensive the new business process is going to be. As explained already, and as has been a general theme throughout this book, we should look at BA from a process perspective. That is why we're introducing the SIPOC (Supplier, Input, Process, Output, and Customer) model.
The model is used to describe a process. What we will do now is to describe a process before and after we've established a BA system, and then sum up: What are the one‐off costs, what are the differences in the cost of driving the process, and what is the added value on the output side—both for the process users and as a saving for the people executing the process?
In the following example, we have decided to carry out a BA initiative in a company that has many employees and is finding it difficult to retain them. Historically, the company has had a process built on BA, based on recommendations from the human resources department (HR; Supplier), having transformed data into some reports in the BA department via a reporting module (Input). This has resulted in a report (Output) that has been delivered to HR and top management (the customer or user of the process). (See Exhibit 8.3.) Reactions to these reports have been sporadic. They have been read, of course, but they have not prompted any direct or systematic actions. It's been more of a case of using the reports as an argument, if they were useful for individual stakeholders in a given situation. The process is described in the middle column, which is typically the first thing done. Based on this, we identify input and output in relation to the process, as well as who the suppliers and recipients are. Another advantage of describing the process in this way is that we clearly define what we are working with and identify all project stakeholders and their roles, influence, and interests.
Exhibit 8.3 SIPOC Diagram Showing the Current Status of the Process
The idea with this new business initiative is that we, through data mining, must identify which employees leave the company, when, and why. With that background, we can initiate retention initiatives, which could be individualized salary packages, education and training, dialogue with management, or considerations concerning the hiring of different employee profiles in the future. Also, demands on individual employees' immediate superiors may be made—supported by bonus systems—with the aim of reducing employee costs by, say, 10 percent per year.
The new process also gets a new supplier, finance, who must deliver continual information about how much it costs to refill vacant positions, and about the costs of the different initiatives (Supplier). This is the reason the finance function is included in the input column in Exhibit 8.4, which describes the new process. Moreover, we must acquire some tools for the continual measuring of the quality of the new process (Input). What we are now gaining from the process is information about which employees historically have left, and which in the near future might be expected to leave. We are also establishing a process of monitoring, which ensures that everyone is working with the same objectives. We are linking some budgets to the different activities that will be initiated so that we, from the operational level, can monitor our resource consumption and, in the long run, analyze what works and what doesn't (Output). The users of the process become not only human resources, who must carry out and plan the activities, and top management, who must evaluate whether the process achieves its targets, but also the employees' immediate superiors, who are both rewarded based on the process and who are responsible for its operative execution (Customers).
Exhibit 8.4 SIPOC Diagram Showing the Process We Want to Create
The process is now such that we, as always, get data about our employees and about who has resigned. Now we start making models, too, which can segment employees by their expected tendency to resign. For each of the critical segments, we analyze their behavior and design campaigns that will retain the employees. The campaigns are implemented, monitored, and evaluated.
What we have described so far is the process as it was and the process as it became. The costs of the business case can be identified based on these descriptions and is linked to the new transformed resources.
In this case, this information is easy to get hold of, as it's all about getting access to some key exhibits from the finance function, which describes the budgets and some costing keys for what costs are involved in re‐employing people for the different types of positions. The transforming resources will cost more, because we need to buy data mining software and train internal users, which are assumed to be one‐off costs. Moreover, costs are involved in training the people who are to use and act on the basis of the new information. Since the solution is within the framework of already existing IT systems, we have no costs of that type.
In conclusion, we can say that it is not one‐off costs that are the heaviest for this business case, since the major costs in connection with this project do not lie in going from the old process to the new process. The biggest costs are linked to the ongoing costs of the new process. Presumably, we will need to use additional human resources in connection with the retention process of employees, and that is an ongoing cost that we have to accept, if we want to make a business case that sums up the ongoing advantages of the project.
In the gray fields, we have outlined benefit statements for the process, which are the value‐creating elements created by the new process. On the left‐hand side, we show that the advantages included in improved control and reduced resource consumption. First, these reflect the fact that we expect to use fewer staff hours for analyzing which employees we lose, in relation to the many meetings and interviews we carried out before. In addition, we expect that the process will keep us updated dynamically, which means that we will be able to react faster in the future. This in turn means that we can reduce the critical time window from the time a need arises among our employees to the time we as an organization can react to this need. Finally, we've got the biggest number—the savings represented by reducing our staff costs by 10 percent per year.
On the right, we list a number of benefit statements for the customers of the process, which are all the company's processes and thereby have an effect on the company as a whole. This is why we haven't included the HR department as a customer here. We might argue that HR will be better off, now that they are receiving more precise information from BA about which employees resigned and why, but these advantages have already been included in the cost reductions in terms of reducing staff costs by 10 percent. However, there are also other results that can be derived from improved working conditions for employees. For one thing, they give better customer relations. This is therefore included as another result of our process improvement.
Based on the identified advantages, we can calculate their value in exact exhibits. Naturally, the person responsible for the business case cannot always put a value on relatively abstract quantities such as the customer loyalty effect. In that case, we will have to ask the customer relationship management (CRM) department.
Exhibit 8.5 sums up where to find the different elements for a business case via a SIPOC model. The white fields contain cost elements, which in this context focus on dividing costs in one‐off costs and ongoing costs, respectively. The gray fields are still showing the advantages delivered by the new process to the company, which for one thing entails that we use the information in a new way. The black fields are not included, but might have contained information about who achieved savings and who gained added value from the process. Finally, the output field will tell us what the new output was specifically, but not its value. That is why it is blacked out.
Exhibit 8.5 SIPOC Diagram Focusing on Costs and Benefits (years)
WHEN PROJECTS RUN OVER SEVERAL YEARS
Generally speaking, business cases for BA projects should be presented with a calculation of the present value of the project. This is due to the fact that the establishment of BA projects, along with their subsequent effects, may have a span of many years. If, for instance, we establish a project that must finance itself over the next ten years, we have a financial outlay that we must take into consideration. We may go out and borrow the money, which means we pay interest to a bank until the project has paid for itself. Alternatively, we may have the money ourselves already, but we tie it up in the project and incur opportunity cost equal to what this money could have earned us had it been invested in another project. Therefore the net present value (NPV) is often calculated for BA projects.
NPV is calculated by discounting all financial costs with an interest rate that cancels out capital costs and is adjusted according to the risk of the project. Only if the calculated NPV of the project is positive should we consider implementing the project.
In the following section, we'll be looking back at the radio station case study in Chapter 1, where we established that the BA project had implementation costs of $1 million and then resulted in additional advertising revenue of $4 million per year. We shall assume that there are no ongoing additional costs related to the project, such as new employees and software. The risk‐adjusted return requirement to the investment was set at 12 percent by the radio station's finance department. The cash flow from the radio station's BA initiative is illustrated in Exhibit 8.6.
Exhibit 8.6 Cash Flows from the BA Initiative in the Radio Station Case Study
The NPV of the project can now be calculated as follows:
In our calculation, we assume that the $4 million is an endless annuity—something that could be questioned, of course. The completed business case shows that the project should be implemented because the NPV is $32.3 million. Note that if the project were shown to be very risky, and the finance department therefore demanded a return of 500 percent, the NPV of the project would be $–0.2 million, and the project should not be implemented. This calculation is made by replacing the interest factor 0.12 with 5 in the previous calculation.
WHEN THE UNCERTAINTY IS TOO BIG
In practice, however, there are many cases in which it is very difficult to produce sound estimates of future cash flows for investment calculations in terms of revenue from BA projects. Cash flows related to costs are usually easier to calculate. Cash flows from investments in bond portfolios can be predicted or estimated with a high degree of certainty. However, if we invest in a dashboard for the management, with key performance indicators (KPIs) for the monitoring of sales processes, the financial implications can be complex or uncertain in terms of comparing and prioritizing projects.
We must then turn to cost/benefit analyses, which are built on arguments rather than exhibits. Such an analysis should indicate whether the project is viable for the organization in relation to its cost and risks.
A qualitative business case based on the cost/benefit method may consist of:
- A descriptive part
- The cost/benefit analysis itself
Let's assume that it was not possible to quantify the revenue‐related consequences of the radio station case study, but possible to quantify only the cost‐related ones. Then we can begin to develop the business case based on the cost/benefit method.
The Descriptive Part of the Cost/Benefit Analysis for the Business Case
In relation to the radio station example, the description could look like this:
- Title: “Know the current listeners' preferences and adapt the broadcast to these.”
- Current status: The current status is that nothing targeted is done to adapt the radio production to current listeners' wishes. It's completely random which news is read and which music is played. DJs frequently attempt to estimate who their listeners are at different times of the day, but it is pure guesswork and not based on factual knowledge.
- The consequences of not implementing: The radio station's production department cannot work in a targeted way to adapt processes to current listeners' preferences with a view to improving the “Average listening time” KPI. The consequence is less than optimal advertising revenue and less than optimal return on equity. In other words, the radio station's production department is not fulfilling its potential and is therefore underperforming.
- Critical success factors: Since it's the first time that a BA initiative has been implemented in the production department, the operational decision makers' change readiness is critical to the success of the project. Another critical success factor is whether we succeed in collecting the desired data about our listeners' characteristics and preferences at different times of the day and whether this data is of the right quality because it was obtained through a questionnaire on the radio station's Web site.
- Target group: The radio station's production department is the target group of the BA initiative, which aims to increase average listening time.
- Risk: As illustrated in Exhibit 8.7, the risk is associated with the data collection via the new data source and the electronic questionnaire as well as the operational decision makers' change readiness. Note from the exhibit that it is not considered likely that these events will occur.
| Risks | Consequence | Likelihood of Event |
| It is uncertain whether the radio station will succeed in collecting the desired data about its listeners' characteristics and preferences at different times of the day in the right quality, via a questionnaire on the radio station's Internet portal. Note, however, that the radio station is budgeting with advertising jobs from sponsors to motivate listeners to fill in the questionnaire on a regular basis and in a qualitative way. | 5 | 2 |
| There is also uncertainty in terms of the operational decision makers' change readiness. | 5 | 1 |
Exhibit 8.7 Risk Involved in the Radio Station's Case Study
Consequence (what is the consequence if the risks of the project occur):
- 1 = No effect
- 2 = Minor delay
- 3 = Delay
- 4 = Considerable delay or drop in value
- 5 = Impossible to implement the project
Likelihood of event (what is the probability of the risk occurring):
- 1 = Very low probability
- 2 = Low probability
- 3 = Sometimes
- 4 = A good chance
- 5 = Almost certainly
The Cost/Benefit Analysis Used for the Business Case
The cost/benefit analysis of the project may consist of an assessment of the eight factors in Exhibit 8.8 before and after implementation. They have been plotted into a radar diagram with numbers between 1 and 4.
Exhibit 8.8 Outline of Benefits
As illustrated by Exhibit 8.8, the BA project is expected to add strategic value to the radio station along with improved competitiveness, improved processes, increased knowledge, and significantly improved measurement of operational processes. An executive brief could therefore look as shown in Exhibit 8.9.
| Costs per Year including VAT | Benefits and Risks in Implementation |
| USD 1 million in the year of implementation. Subsequently, there will be only marginal maintenance and training costs related to the information system. | The BA project is expected to add strategic value to the radio station along with improved competitiveness, improved processes, increased knowledge and a significantly improved measurement of performance in terms of operational processes. Customer relations are also expected to be improved. Moreover, data quality is expected to be significantly improved, since we no longer have to guess who our listeners are. No benefits are added to the technological platform, and it therefore remains unchanged. Risk is associated with data collection and the operational decision makers' change readiness. It is, however, considered unlikely that any of the risk components will occur. |
Exhibit 8.9 Cost/Benefit Analyses for the Business Case
PROJECTS AS PART OF THE BIGGER PICTURE
In BA we have a rule of thumb that says: Think big, start small, and deliver fast. This obviously means that we have to look at our projects as part of a bigger picture, and to this aim, maturity models are useful tools. As mentioned previously these models are a firm fixture in the business concept of most IT solution vendors, and they do have a number of advantages.
First, they are able to place individual information systems in a greater context (i.e., we can make a development strategy at the information system level and describe the business opportunities that it opens). If we are talking about, for instance, CRM processes, it's difficult to generate campaigns if we do not have a data warehouse from which to draw information. It can be done, but with difficulty, and data quality often suffers. If a data warehouse has been established, we are able to design individualized campaigns if we involve analytical competencies. The campaign will typically be of an added‐sales or customer retention nature, where we are looking to optimize customer lifetime value. After we've established our information wheel, we want to try to improve its effect by constantly optimizing the process via automation (which makes it cheaper) and by increasing the relevance of the messages we send our customers (more relevant content at a more relevant time), which is the idea behind pervasive BA. In CRM, this is called marketing automation. The idea is that when a customer changes his or her address, for example, information is automatically sent to the customer about where to find the nearest local store in his or her new neighborhood. Or, if a customer usually buys a new phone around Christmas, we automatically send this customer a relevant offer so that he or she does not even have the time to go check out our competitors.
Exhibit 8.10 gives a generic outline of what a maturity model might look like for an information system. As is also shown by the model, we use terms such as revolutionary and evolutionary developments of systems: a revolutionary development takes place in connection with information systems being upgraded to a technically higher level, whereas an evolutionary development takes place in connection with users of the technical solution learning to master it and internal processes being adapted to the new opportunities.
Exhibit 8.10 Revolutionary and Evolutionary Maturing of Information Systems
It is also inherent in the model that the development must take place in phases, with each based on the previous one. For example, it doesn't make sense to implement a marketing automation system if we do not have a data warehouse on which this system can base its actions and in which campaign responses can be collected. Similarly, it's quite difficult to carry out data mining if we do not have a data warehouse, where the information with history is stored.
In Exhibit 8.11, we've made a table with the same four maturity levels that are presented in Exhibit 8.10. What we've done—true to the principles of this book—is to divide the information system into technology, processes, and competencies and generically describe the levels.
| Maturity Level | Focus Areas and Characteristics | Characteristics of the Information System | Processes | Competencies |
| 4 |
|
|
|
|
| 3 |
|
|
|
|
| 2 |
|
|
|
|
| 1 |
|
|
|
|
Exhibit 8.11 Generic Maturity Model for an Information System
The purpose of a maturity model for an information system is to be able to put it into a greater context, such as making a development strategy for the information system.
This also means, of course, that we can analyze which elements we need to be aware of in connection with the development of the information system by asking questions such as these:
- Do we have the people skills we need?
- Will the person responsible for CRM accept our going from a creative to a fact‐based decision process in connection with the establishment of new loyalty campaigns?
- Do we have analysts with solid and relevant business insight to perform process optimization in connection with marketing automation?
A maturity model gives us the opportunity to identify the critical success factors that we need to focus on to make sure that our technical investments deliver a positive return.
Finally, maturity models give us an opportunity to relate technical solutions to strategic requirements. If we want to increase our customers' loyalty based on historical customer behavior, then we will have to establish strong analytical competencies and, for example, a data mining solution. Moreover, we are able to, with a point of departure in the knowledge acquired about competitors' processes, gain an insight into which information systems they are using, and thereby how they use information as a strategic resource. In continuation of the strategic perspective, a maturity model also enables companies to consider where the market is going to be in five or ten years' time. If, for instance, we're feeling pretty sure that the industry we're in is characterized by everyone using marketing automation in the future as a cheap and effective means of creating loyalty, then the question is not whether we should invest in this solution. Because we obviously have to avoid losing customers, it's purely a question of when to invest. Consequently, whether we want to be the market leader in this field or wait in the hope that the implementation costs will drop, we will have make the investment before we've lost too many customers by having CRM processes that are below market standards.
Case Study on How to Make an Information Strategy Roadmap
A global telecom operator asked us to help them define how they should compete on information five years from then. It was described as a target scenario for how their analytical competencies should be in the future and how to get there. The driver of this project was the IT department, as IT felt that they had many capabilities to offer the business side of the organization. The issue was, however, that the current dialogue was not existing on the right governed level to make it happen. In steps the project was executed as per below:
- To manage such a large scope process, the obvious first step was to set up a strong project organization with the CIO and the CEO as sponsors. Then a few days would go into planning how to run this project, getting the buy‐in from all stakeholders of the suggested approach. This would include getting a clear scope for what the deliveries were, as well as designing a mission for what we were about to achieve, including getting all project members to agree to this mission statement.
- The next step was to fully understand the company strategy. As strategies can be formulated in many ways, it was important for the project to understand the company's ambition to create full alignment.
Also, because strategies are often formulated as a series of projects that must lift the processes to a next level it is important to understand why the processes must be lifted to this new level. Strategies typically only look one to three years ahead, but the IT capabilities must be thought even further ahead as in the case of this information roadmap. Finally, this strategic knowledge was essential for the team during the interviews with the functional and process leaders, making sure that they were properly engaged and challenged on the visions.
- The next task was to focus where we could make a change. In other words, there are some natural focus areas within organizations defined by the strategy. These focus points are typically also where change is going on anyway, making it easier to implement information driven changes with a significant impact.
- For each of the focus areas, stakeholders would be engaged. This could, for example, be the fixed net division, where the strategic requested change primarily would be about how to work better with customer loyalty. Hence the focus would be on the sales and marketing department, while at the same time this meant disregarding, for example, cable planning processes where analytics also could be have been used.
Together with each of these departments, we would identify their key processes as per strategy and make individual maturity maps for them. For example, a marketing department could have five key processes; innovate offers, create value proposition, execute campaigns, internally optimize the campaign landscape, and measure the campaign effects as per targets. The maturity maps would also have information maturity dimension always based in five levels: not using data, using fragmented data, using data warehouse information, using analytics, and using real time functionalities.
When developing these maturity maps per function and its processes, we would then agree with process owners on where the functions were at the time and where they should be in the future, including all the things that must be done over time to get there. Things that had to be done could include, for example, improving data quality, providing better front ends, training users, getting CRM concepts and customer journeys in place, and the like. Each of these things that needed to be put in place would be described on one pager called a Business Idea In Brief (BIIB). It is here important to realize that a BIIB can be process specific but also more general for many processes (e.g., improved data quality, which will affect many business processes ranging from marketing to sending out bills).
- The following step would be condensing all these business ideas in such a way that they are mutually exclusive and fully cover what must be done. Also, it is necessary to group them into natural clusters so that the stakeholders who must prioritize the more than 200 BIIBs could get an overview.
- The next step would then be the prioritization workshops at which every BIIB would be presented; stakeholders could either accept them (possibly adapting them on the fly) or reject them.
- The following step would be to put all the BIIBs into roadmaps as individual projects, looking for synergies between divisions. For example, if more divisions wanted real‐time marketing functionalities, there could be synergies. In addition, the sequencing would consider what had to be done first (e.g., data quality comes before real time analytics, since imprecise real time analytical models would just mean sending irrelevant or imprecise information out into the market). We, so to speak, have to deliver information maturity from the bottom and up of the maturity map. Again, the road maps have to be signed off by a stakeholder before being sent back to the strategy office for funding procedures.
- The final step of the exercise was to create a governance around the analytical road map. Where the governance would be around quarterly meetings in which IT and the business side would make reviews in regard to whether IT had delivered what they should and whether the business side was actively using the capabilities as per strategy. Finally the governance would also include a continuous updating of the information roadmap, making sure that it was not only a one‐off exercise but a continuous journey based on dialogue between the IT and the business side into the information age.
This approach has, by the way, been used for many public and private organizations over the years, as well as down to the information needs of individual departments. For more, go to BA‐support.com, where there are examples of the templates and a more detailed program description.
SUMMARY
In this chapter, we've presented different approaches to prioritizing different project initiatives. Since there are quite a few things to take into consideration, we've made a decision tree for inspiration (see Exhibit 8.12). The tree also sums up what was covered in this chapter, namely that we always start by determining whether a given project is of a strategic nature. If the answer is yes, our project is based on strategic arguments and should not be prioritized based on a business case. We must expect this prioritization to have happened at a strategic level. We should, however, make sure that the project has been given a budget and that we've got the full support of the business.
Exhibit 8.12 Chapter 8 Mind Map
Over a considerable amount of time, large projects will take up a lot of resources, and we will therefore try to place them in a strategic context. If we don't succeed in this, we should consider whether these projects justify the potential opportunity costs involved, as they will reduce the agility of the BA function for a long time.
If we are able to come up with a reasonable estimate of the value creation of a project, this is to be preferred; otherwise we must make a qualitative assessment of the value of the project. And finally, whenever possible, carry out precise cash flow analyses that cover the entire lifetime of the projects, and adjust capital costs if they run over multiple years.