From Descriptive to Diagnostic, Predictive, and Prescriptive Analytics

Basic BI implementations are everywhere, in every corner of every organization. They range from personal spreadsheets to financial and production reports to executive dashboards. The sprawl of these applications, particularly as a result of commonplace data warehouse implementations over the past twenty years, no doubt has improved enterprise transparency and influenced improvements in productivity, customer and partner relationships, and compliance. But slicing and dicing and reporting on information, in most cases, has fallen woefully short of producing measurable economic benefits. And where these implementations may have been measurable, few organizations have actually measured them other than with poor proxies of value such as “user satisfaction.” Ultimately, BI and data warehousing have become a significant IT cost sink, in many instances only with acknowledged soft and unmeasured business benefits. Yet inertia to continue generating hindsight-oriented reports and dashboards is a function of having chased after them the past twenty or thirty years.

Typical BI implementations tend to be far removed from actually monetizing data, or connecting the dots between them and top- or bottom-line value propositions. More advanced analytic capabilities which generate actionable insights, predictions, or explicit recommendations can be connected more readily to economic improvements. Therefore, there’s an imperative for organizations to reach for advanced analytics capabilities (Figure 4.1).

However, advanced analytics initiatives tend to be more challenging and vocational—that is, targeted at a particular business problem or opportunity rather than “the enterprise.” And because information reporting and exploration continue to serve a valid purpose in organizations (especially with the emergence of self-service BI), it can be helpful to consider BI and advanced analytics as distinct entities and initiatives with unique value propositions, staff, and technologies from one another.

Figure 4.1 Gartner Analytic Ascendency Model

The Advanced Analytics Advantage

Fifty years after Coca-Cola acquired the Minute Maid brand in 1965, a customer called to complain about the consistency of orange juice from week to week and from one season to another. Except this wasn’t just any customer. It was a popular fast food restaurant with over two thousand locations. They had threatened to switch to another “orange drink” provider.³

If there’s anything Coca-Cola prides itself on, it’s the quality and consistency of its products. Oh, and not losing customers. So this call set into motion a project to determine just how to establish orange juice consistency regardless of the season, supply chain disruptions, or regional orange prices and availability. Our job was to “take Mother Nature and standardize it,” said Jim Horrisberger, director of procurement at Coke’s massive juice packaging plant an hour south of Disney World. “Mother Nature doesn’t like to be standardized.”⁴

But that’s just what Coke did. With the help of Revenue Analytics, a consultancy that also helped Delta Airlines maximize its revenue per mile, “We basically built a flight simulator for our juice business,” said Doug Bippert, Coke’s VP of business acceleration. The so-called super-secret methodology called “Black Book” analyzes a quintillion decision variables including weather patterns, anticipated crop yields, and cost pressures, plus the six hundred flavors Coca-Cola scientists determined comprise an orange. Then the algorithm dictates how to blend and batch oranges, based on their flavor analysis, for consistent juice.⁵

Now, Bippert said, “If we have a hurricane or a freeze, we can re-plan the business in five or ten minutes because we’ve mathematically modeled it.” So the next time you drink orange juice, remember you’re drinking an algorithm.⁶

Whereas BI implementations are appropriate typically for informing business managers of performance indicators, advanced analytics implementations can provide far-reaching organizational benefits over basic BI. The requisite BI tool/platform features of data aggregation, summarization, selection, slicing, drilling, and charting are tuned for presenting information interesting to users, but not necessarily information important to optimizing business processes. Today, only strategic decisions (and not even all of them) may be made at a rate slower than the speed of business. Tactical and operational decisions increasingly must be made at a rate faster than that of which humans are capable.

Despite continued corporate reliance on spreadsheets, now analytics is a core competency in most organizations. However, most implementations—particularly enterprise implementations—entail basic decision support solutions for business managers and executives. In addition, much information is still left to interpretation, pockets of analytics result in information clashes between departments, and many users choose not to rely on analytic output to guide their decisions and behavior, or at least limit their reliance on it. Resolving this demands producing analytic results beyond simple summarizations, and involves delivering those results direct to business processes–not necessarily to people. Directing analytic output merely at eyeballs continues to be one of the great fallacies and limitations of BI.

However, advanced analytics in all its variety of instantiations (for example, data mining, prediction, artificial intelligence, complex-event processing, visualization, and simulation) can be not only difficult to implement, but also difficult to articulate and coordinate. This is why even the most well-intentioned analytic initiatives can too easily devolve into a pedestrian presentation of lagging and loosely relevant performance indicators.

Ultimately, the keys to generating measurable economic value with information involve implementing forms of advanced analytics focused upon:

• Exploiting Big Data. Increases in the volume, velocity, and variety of information assets are a boon for data monetization. Analytic capabilities have struggled to keep up with the size, speed, and range of available information. But organizational capabilities are just as important.
• Improved and actionable decision making. Analytic technologies and solutions can drive improved and actionable decision making, leading to increased business performance. Solutions for decision making tend to embrace complexity, including sophisticated scenarios. And they frequently involve improving or automating business processes, or reducing risk.
• Identifying monetizable insights. In addition to classic decision making and automation, monetizing information through advanced analytics can be achieved through deeper understanding of processes, people, and markets; improved foresight; and innovating products and services.

While many of the ideas you conceive will overlap some of the above, each of these should be a starting point for exercises in identifying ways to generate improved economic benefits available information. Ask yourself and others in your organization: How can we exploit the characteristics of Big Data? How can we use information to make better/faster decisions? What questions can we ask now that couldn’t be addressed earlier? The answer to each of these most certainly will lead to opportunities for monetizing your (and others’) information assets.

Monetizing the Increased Volume of Information

With regard to Big Data volumes, basic BI solutions tend to be challenged. But advanced analytic solutions, particularly data mining, machine learning, and predictive applications, flourish with large datasets. The larger the dataset, the easier it is to uncover hidden subtle patterns and trends, and therefore, opportunities which are likely to evade the competition.¹⁰ In addition, the subtle signals are magnified by large datasets, and a larger breadth of dimensions can be established to discern business monetizable correlations. If the goal of a business process is to increase the probability of a customer transaction happening, then having a wealth of data about, for example, that customer, similar customers, market/shopping conditions, and product and upstream supplier inventories, along with subtransactional data¹¹ about the customer’s real-time shopping behavior (online or in-store), can be used by advanced analytic applications.

One excellent example of a company taking advantage of massive volumes of data is Vestas. Founded in 1898 as a blacksmith shop in western Denmark, Vestas has breezed into a global leadership position in the wind turbine business with over 75 gigawatts worth of generating capacity worldwide.¹² To optimize the placement of wind turbines Vestas thus far has collected about 20 petabytes of weather data. This includes information on temperature, barometric pressure, humidity, precipitation, and wind direction and speed from altitudes of ground level up to 300 feet at the tip of the blades. Feeding this information into an IBM supercomputer has enabled Vestas to reduce wind forecast modeling from three weeks down to fifteen minutes, and precisely optimize the placement of turbines within a 10-square-meter grid—whereas previously it could only optimize placement within a 27-square-kilometer grid. With the ability to develop turbine sites one month faster, Vestas installs one turbine every three hours somewhere around the world, passing along these savings and margins to customers, employees, investors, and product development.¹³¹⁴

Monetizing the Increased Velocity of Information

As with the earlier Lockheed Martin project insights example, the increased velocity and variety of project information made available enables it to anticipate complex design and development program issues much earlier, thereby saving millions of dollars in cost overruns and rework.

Typical BI solutions where analytic output is directed at people not processes can falter as the speed of data inputs ratchets up. Increased information velocity implies an increased speed of business. At even low levels of velocity, humans become incapable of ingesting and using information efficiently, at which point business process effectiveness can suffer. This is the point at which advanced analytic applications are required that consume and respond to swift streams of data (think of algorithmic stock trading.) These apps can make recommendations either to users responsible for administering an operational business process or be integrated with the business process applications themselves.

Monetizing the Increased Variety of Information

While many BI solutions can report on information from multiple sources on demand, or make use of integrated data in data warehouses or marts, they don’t truly take advantage of this diversity of data. A query tool can readily co-present data from differing sources, but tends to falter at corelating in ways other than standard regression analyses. Advanced analytics technologies, however, specialize in relating multiple data sources to discover unforeseen relationships such as matching employee candidate qualifications to the skill sets and experience of an organization’s best employees.

By 2:1 over the issues of information volume and velocity, IT and business professionals contend that the variety of information not only is the greatest information management challenge, but represents the biggest opportunity for generating economic benefits from information.¹⁵¹⁶

This is why organizations like Coca-Cola discussed earlier find huge measurable benefits in integrating information on crop yields, cost pressures, GPS data, and the six hundred flavors comprising an orange to optimize orange juice production consistency.

Even public sector organizations can find ways to monetize a variety of information. Pinellas County in Florida implemented the Centers for Disease Control’s Youth Risk Behavior Surveillance System (YRBSS) which monitors six types of health-risk behaviors considered leading causes of death and disability among youth and young adults. It integrates data from systems which track injuries and violence, sexual behavior and STD rates, alcohol and drug use, tobacco use, dietary behaviors, level of physical activity, and the prevalence of obesity and asthma. The youth risk index then is used to implement one or more programs within three areas: parental engagement and parenting practices, school connectedness, and/or protective services. Through improved understanding of these issues and the factors which drive them, Pinellas County has been able to make strides in reducing the unfortunate expenses associated with them.¹⁷

Increased information volume, velocity, and variety characteristic of Big Data can also help magnify, identify—and thereby monetize—hidden nuggets of insight.

Embracing Complexity, Unexpected Activity, and Changing Conditions

In addition to embracing complexity, advanced analytic solutions can handle unexpected data and dynamic business/market conditions more effectively. Machine learning algorithms, and neural networks in particular, make sense of more randomized inputs, adeptly dealing with outliers or information that doesn’t fit otherwise prescribed formats, categories, or time frames.

Until recently, an Australian insurance company had segmented customers on six dimensions using an off-the-shelf BI tool. This led to insurance products and marketing campaigns which failed to differentiate the company or gain market share. After supplementing its BI implementation with an initial deep analysis of customer data, resulting in an awareness of more than one hundred significant dimensions (including artificial derived ones), the company was able to identify and target gaps in the market quite easily.

One of the most dynamic activities in any business is the configuration and evolution of complex products. At some point NCR business leadership realized its ATMs were offered with tens of thousands of configuration options. Dealing with this kind of SKU proliferation on top of increasing product complexity adversely affected the sales cycle. Across the value chain, NCR experienced a lack of alignment between customer demand, solutions management, operations, and sales. So with the help of analytics software from Emcien, it produced a demand-shaping pattern analysis for determining the optimal number of product configuration options, resulting in a $110 million bump in revenues and a 5 percent increase in sales efficiency.¹⁸

Optimizing Business Processes

Ultimately, any form of information monetization is the result of some business process or combination of business processes. BI tools generally are standalone with respect to the business processes that they support. Even when embedded into business applications, they tend to present charts or numbers in an application window. Ideally, output is updated to reflect the user’s activity and needs, but less often is it used to affect the business process directly. Evolving to complex-event processing solutions, recommendation engines, rule-based systems, or artificial intelligence (AI), combined with business process management and workflow systems, can help to optimize business processes more directly, either supplementing or supplanting human intervention.

Case in point: a company formed from a collection of shopping stalls in 1919 by an English trader named Jack Cohen today has hardwired its thousands of refrigeration units to a data warehouse. By 1996, Cohen’s company would become the United Kingdom’s largest retailer, collecting £1 of every £7 spent in the U.K.¹⁹ Tesco now has over 3,100 stores across the U.K., Europe, and Asia.

“Ideally, we keep our refrigerators at between -21 and -23 degrees Celsius, but in reality we found we were keeping them colder.” said John Walsh, Tesco’s energy and carbon manager for Ireland. “That came as a surprise to us.” In a project collaborating with IBM Dublin’s research laboratories, Tesco worked with refrigeration manufacturers to collect data from in-store controllers and feed it into a dedicated data warehouse. The system takes unit readings twenty times per minute and overlays the analytics on a Google map illustrating their performance. Each unit produces seventy million data points per year. With sophisticated statistical processing and machine learning, the system can identify refrigeration units which are performing suboptimally, even when knowing nothing about how refrigerators should perform. As a result, upon complete rollout, Tesco expects to deliver €20 million per year to its bottom line via a 20 percent reduction in refrigeration energy expense, and eliminate 7,000 tons of CO 2.²⁰

Automating Governance, Risk, and Compliance

Even before mandatory governance, risk, and compliance (GRC) reports are produced—typically with BI or corporate performance management platforms—organizations should have the means to identify potential, imminent issues before they occur. Advanced analytic applications, tied to data streams consisting of leading GRC indicators, can make business process owners aware before trailing indicators.

Deeper analytic capabilities are needed to uncover the ultimate source of discovered GRC or fraud issues through the forensic examination of transactions and other business activity, including contrasting those activities with an understanding of written contracts. Additionally, as a matter of course during prescribed audits, robust analytic tools can go beyond the manual sampling still done by most auditors today to assess every record fully.

One of the biggest risks anywhere is driver fatigue, especially for those driving big rigs. “Trucking and logistics is an extremely challenging and competitive industry,” explains Lauren Dominick, Predictive Modeler at FleetRisk Advisors, a 25-person commercial fleet consultancy based in Alpharetta, Georgia. “These companies need to be able to deliver goods reliably and on schedule, while ensuring driver safety and avoiding accidents that can cause delays, damage cargoes, and even cost lives.” Dominick also describes how the work environment is stressful with an average employee turnover rate exceeding 100 percent annually.²¹

The idea behind FleetRisk Advisors is one of pure information monetization: that companies have a mountain of unused information (dark data) about their drivers and vehicles. Originally, FleetRisk’s process suffered under the strain of manual calculations and basic types of reporting. Today, it automatically compiles near real-time telematics data from each driver’s electronic logbook and integrates it with employee information from other systems. With this integrated information asset, FleetRisk analyzes drivers’ pay versus one another and industry averages, and combines this with other stress factors and employment history. As a result, it has been able to help reduce employee attrition by more than 30 percent saving on training and other hiring-related costs. More important, explains Dominick, “Across our client-base, we’re seeing a minimum of 20 percent reduction in the overall accident rate, and an 80 percent reduction in severe accidents such as rollovers, driving off the road or rear-end collisions. By identifying the risk factors—especially those that contribute to fatigue—we help our clients to intervene before accidents happen.”²²

Enhancing Scenario Planning

Even beyond forecasting and other kinds of prediction, advanced analytic capabilities can include comparative risk-reward assessments (for example, Monte Carlo simulations). This is the realm of sophisticated economic modeling. Yet most business leaders are barely aware of such methods, let alone trained in them. Increasingly it is becoming critical for organizations to embrace advanced analytical solutions like scenario planning that can be used to drive optimal organizational strategies and performance.

One planning process we all know could use some improving is the efficiency of airline gate and ground personnel. But the problem isn’t with the personnel themselves—rather, it’s with their scheduling. It’s a little known fact that estimated times of arrival (ETAs) are provided by pilots themselves. 10 percent of the time these estimates are off by more than ten minutes, and 30 percent of the time they’re off by five minutes. Pilots may know their speed, altitude, direction, and distance to the airport, however they lack other detailed information and the analytic capability to provide more accurate estimates.

But a system being installed at airports around the world developed by PASSUR analyzes information about weather, fl ight schedules, and historical flight data, plus data feeds from its network of passive radar stations installed near airports. This immense multidimensional body of information enables the computation of refined ETAs by evaluating current conditions versus prior “sky scenarios” to model and balance likely outcomes and risks. At airports where this system is in place, the improved efficiency of personnel scheduling and throughput can yield several million dollars per year. And at such outfitted airports, United Airlines claims its fleet is avoiding two to three airport diversions per week.²³²⁴ At a cost of up to $200,000 each, sky scenario planning is a significant way to monetize this information.

Understanding Unstructured Information

BI technologies and implementations are inherently hard-coded to accept certain kinds of data from known sources in understood formats. Advances and extensions in BI technology have made it possible to perform rudimentary analysis on unstructured text. But when it comes to truly analyzing it using language and syntactically aware algorithms, graduating to advanced analytic technologies which specialize in this is required. And beyond written text, sources of enterprise intelligence and performance increasingly are in even more complex formats, such as audio and video. As discussed in chapter 2, much of this unstructured information lies dormant in the form of what’s called “dark data.” Much like the dark matter of the universe, we know dark data is there because it has a gravitational effect on our business and operations as it remains unused and unmeasured.

As I have long advised clients, there are only three ways to generate economic benefits from unstructured information (or content) before you structure it: you can read it, search it, and sell it. But the process of extracting most forms of value from unstructured information, even the process of categorizing or parsing content, implies some form of pre-processing. Maximizing the economic value of most content involves analyzing it in some way, either to identify patterns or structure it so as to render it as input to some automated process.

This is just what the Birmingham, Alabama-based insurer Infinity Property and Casualty did to identify indicators of fraudulent insurance claims. Infinity’s senior vice president of claims operations, Bill Dibble, doesn’t believe in “silver bullets”—but he had this idea that insurance claims could be scored using predictive analytics in much the same way as credit applications. While Infinity already had a rudimentary system which screened questionable claims based on “red flags,” it still required quite a bit of manual intervention, thereby slowing down the claims process, and affecting customer service. Moreover, its hard-coded flagging process had trouble catching emerging fraud patterns. However, by text mining the content of previous claims known to be fraudulent, Infinity matches these patterns to the content of police reports, medical files, and other accident-related documents. These patterns of “narrative inconsistency” indicate probable fraud.

With this new predictive claims analytics system in place, Infinity’s success rate in pursuing fraudulent claims jumped from 55 percent to 88 percent, and in the first six months of operation the system increased subrogation recovery by $12 million. In addition, it has reduced its cost of claims adjustment, leading to improvements in its Loss Adjustment Expense (LAE) ratio. As Dibble exclaimed, “With predictive analytics, we were basically able to close a hole in our pocket where money was leaking out steadily.”²⁵²⁶

Identifying Faint Signals

Competitive advantage and sometimes saving lives increasingly relies on the ability to identify and monetize faint signals. These are signals that either present themselves with infrequent occurrence (for example, fraud, missing input) or in subtle patterns (for example, supplier quality glitch, inventory leakage, and underserved market subsegments). However, human fingers and eyes clicking through and viewing data with a BI tool can all too easily miss these signals. We are at a tipping point in the evolution of the Information Age. Decision making increasingly is automated and managed by machine-based models far more effectively than any human could manage.

Illustrating these faint signals is the web crawling and analytic technology behind HealthMap, a system that really got on the map in 2014 when it identified a clustester of “mystery hemorrhagic fever” cases in Guinea. It wasn’t until ten days later that the World Health Organization, having been notified by the health ministry of Guinea a day earlier, announced an outbreak of Ebola. Developed by researchers, epidemiologists, and software developers at Boston Children’s Hospital, HealthMap scours billions of posts from tens of thousands of social media sites, local news, government publications, infectious-disease physicians’ online discussion groups, RSS feeds, and other sources to detect, plot, and track disease outbreaks. Every hour the application repeats the process, analyzes the text in fifteen languages, filters out noise such as “an outbreak of home runs” or “Justin Bieber fever,” and determines the geographic location referenced in the content. It also incorporates a machine learning algorithm which improves over time—matching confirmed disease outbreaks with the information it collected on them, and learning to anticipate their spread.

“Getting case information and data of very specific epidemiology, of outbreaks, and fit that into disease models, where it might spread to, how bad it might be, then integrate it with climate data or transportation data, all of a sudden it’s data you wouldn’t necessarily have access to,” said John Brownstein, epidemiologist and HealthMap co-founder. The ability to quickly identify outbreaks can help local health organizations respond to and contain them faster, not just saving lives, but also saving these organizations and governments money and resources.

To support the cash-strapped WHO in deploying scarce resources, HealthMap continued to track the disease across nearby West African countries.²⁷²⁸ Currently, HealthMap is helping track and now predict the spread of the mosquito-borne Zika virus outbreak.²⁹

Evolving to Insight and Foresight

BI tools are designed to present historical data, not so much assist in looking into the future. Slicing and dicing and drilling data, as is the specialty of BI tools, also can offer users insight into not just what occurred but why it occurred. But ask any stock trader, for example, which is more monetizable—knowing the past, or anticipating the future. The answer is clear.

True, even commonplace spreadsheet software includes a host of statistical formulae and chart types to expose and illustrate trend lines. As is their exclusive domain, advanced analytic technologies greatly exceed basic BI tools in the types of visualizations, pattern matching, and correlative algorithms, predictive techniques and modeling capabilities, simulation, forecasting, and scenario-planning capabilities available to organizations.

Just ask Mike Higgins, manager of sales, analysis, and strategy at Advanced Drainage Systems, whether it’s better to know the past or have a jump on the future. Headquartered in Hilliard, Ohio, Advanced Drainage Systems (ADS) is the world’s largest manufacturer of products for stormwater management and sanitary sewer applications. Previously, it would draw upon reports from operational, sales, and marketing systems such as its Oracle ERP implementation to estimate forecasts, do planning, and set budgets. Then it evolved to pulling this data into an Excel spreadsheet.

“We look at how sales and order activity is performing currently, match that with a five-year historic trend of the seasonality of the business on a month-to-month basis, and produce a rolling twelve-month forecast,” said Higgins. To supplement and tweak the model, “We seek input from our sales managers, based on what they know and data from field intelligence about different types of business activity,” he explained. But that wasn’t enough. Forecasts were still less than 80 percent accurate, often either leaving money on the table or product in the warehouse.

The next step of ADS’s analytic evolution involved working with the advanced analytics company Prevederé Software to assess over one million external datasets for their relevance in enhancing the forecasts. With this access to weather data, global, and local economic data, along with building and construction industry data, ADS identified external leading indicators for its business, creating over 150 predictive models in six weeks. By stripping the bias out of their forecasting and being heads-up to a multitude of external factors, ADS improved its monthly forecast accuracy to 98 percent, leading to a 26 percent increase in monthly year-over-year revenue and a 20 percent growth in monthly sales.³⁰

Instigating Innovation

Beyond basic BI, a market is emerging for analytic solutions that drive business innovation. The advanced analytic capabilities of identifying new patterns, performing simulations, and exposing weak signals previously mentioned can be used to spur novel ways of looking at the business and its operations, strategies, emergent customer needs, and market shifts. As organizations struggle to squeeze yet more cost out of the business, the upside that innovation offers can—and should be—driven by a deeper and more exploratory analysis of available data from within and externally.

Clinical pathways themselves are a fairly recent innovation in the history of medicine. Clinical pathways (or “carepaths”) are used by health care providers to ascribe the set or cycle of recommended tests and treatments and waiting or recovery periods for a particular patient or type of medical condition. According to OpenClinical.org:

As databases of electronic medical records (EMRs) are quickly replacing years of pen-and-paper-based diagnoses, lab results, surgical records, and notes from doctors, nurses, physical therapists, etc., this wealth of data is a fuel spawning additional innovations in health care—particularly mining it to develop new or improved clinical pathways. The imperative is there too. In the U.S., Medicare, Medicaid, and the Affordable Care Act (Obamacare) each incentivize health care providers for the quality of care, and disincentivize repeat visits by a patient for the same ailment.

With this kind of innovating in mind, Mercy Hospitals, one of the top five health systems in the U.S., worked with advanced analytics provider Ayasdi to identify situations in which clinical variation could be narrowed, and narrowed toward protocols which provided better outcomes. The topological data analysis solution, leveraging existing computational biology algorithms and machine learning, surfaces groups of similar patient procedures and generates clinical pathways that result in improved patient outcomes and reduced expense. It pinpoints treatments, nursing orders, patient comorbidities, prescriptions, and even equipment that contributed the most to variations in cost and outcomes. And it can be used to model proactively the impact a care path change will likely have.³²³³ According to Mercy’s VP of clinical informatics, Dr. Todd Stewart, the adoption of clinical pathways increased more than 20 percent by Mercy physicians, clinical pathways are deployed more quickly, and over forty new pathways have been defined covering 80 percent of all clinicians. All told, Mercy is saving more than $18 million per year, and increasing as more pathways are incorporated.³⁴