Decision Matrix

One of the most important tools we use throughout the Analytical Cycle is the Decision Matrix. A decision matrix reflects the outcome and values of various decision scenarios in a grid format. The decision matrix is a great tool to use when looking at a large group of decision factors to assess each factor's significance. This format enables the manager to quickly analyze relationships between the decision factors to determine the optimal choice.

Acts, events, outcomes, and payoffs are the four building blocks of decision theory. Acts are the actions or decisions that a person may take. Events are the occurrences taking place, usually with a level of uncertainty. Outcomes are the results of the occurrences, and payoffs are the benefits the decision maker receives from the occurrences. The benefits within the matrix can be numerical or descriptive in nature.

For example, we are trying to decide if we should drive for no cost or take the car service Uber for $11 to a show. At this particular venue, the parking is free but there are often delays that may make us late for our show. The two decisions we have to choose between are to drive ourselves or take Uber; these would be the acts. The parking delays are the events that cause a level of uncertainty. The payoffs in this scenario are the $11 spent on Uber versus driving ourselves for $0. An additional payoff is to be on time for the show. Here is our decision matrix:

From this decision matrix, we can see that paying the $11 would reduce our uncertainty of making the show on time. The question is whether $11 is worth reducing the uncertainty.

To make a good decision matrix, there are a few principles that we would recommend:

• Actionable—Among the core components of a decision matrix are the acts or decisions that we outline for the user. You need to make sure that decisions you are considering are actionable for the organization, particularly if you are in a large matrix organization. If the actions you outline are too difficult to implement, the analytical solution will probably not get off the ground.
• Rationalize Attributes—How many attributes or metrics do you need to show in your decision matrix to help someone make a decision? This may be one of the toughest questions to answer. While a natural tendency is to consider as much information as possible, we need to ask ourselves which metrics really drive the decision and focus on these.
• Payoff—You must determine a monetary value or some other utility derived from making the decision, like an on-time arrival to the show. This is the incentive to make the decision. In most of our work, we find adding a monetary value to the decision helps the analyst weigh the economic benefits versus the risks, leading to better results.
• Publish Your Math—Trust in the techniques deployed will be important as users want to know if your calculation for a particular metric matches their understanding. The simplest way to avoid any misunderstanding is to be transparent.
• Publish Your Sources of Data—Along with the calculation for a particular metric, the source of the data is another area to create visibility around. By letting users know the source of the data for your analytical solution, they can have a better understanding of your results.
• Probability—Adding a confidence factor to an act or decision helps an executive weigh the risk of uncertainty against the payoff. We will discuss this topic in detail due to its high importance.

Probability

The use of a probability or confidence factor can play a big role in a decision. It may be one of the most impactful Lego pieces you can use in building your solution. Probability is the likelihood of an event occurring; the higher the probability the more likely the event will happen. If you know that your next business decision has a 95 percent probability of being successful, it gives you greater confidence in making the decision over one that has only a 20 percent probability.

To show the impact that probability can have on your decisions, let's look at an article by V. Kumar, Rajkumar Venkates, and Werner Reinart, “Knowing What to Sell, When, and to Whom.” They outline several techniques to leverage probability along with the pros and cons. Describing a method they developed called the Customer Probability Cube that generated significant returns for one of their clients, they write, “At the B2B firm, the new methodology increased profits by an average of $1,600 per customer, representing an improvement in ROI of 160%. Given the sample size of over 20,000 customers, the increase in profits amounted to about $32 million for the sample group alone.” This is a great example of the power of probability and its influence on making the right decisions.

While there are several ways to represent probability, we employ a probability score or a descriptive measure. A probability score is an assigned numerical number attributed to an outcome, such as 95 percent. A descriptive measure may be the use of High, Medium, Low, or a relative score of 1 to 5 where 1 is difficult to achieve and 5 is easy to achieve.

Let's look at an example of a relative score of probability. In this fictional example, we are the Edison Credit Card company that sells our credit cards via online channels with no physical presence (see Table 8.1). Our marketing department has developed a decision matrix with the acts, events, outcomes, and payoffs. From this decision matrix, we would probably focus efforts on the Midwest and the Southeast given their ROI.

To help determine the best decision, working with our data scientist, we create a confidence factor for each of the segmented email campaigns on a scale of 1 to 5, where 5 is the easiest to achieve and 1 is the hardest. The ability to achieve takes into account current market penetration, overall saturation of the market with competing credit card companies, number of potential buyers that fit the credit card companies' target offer, or other factors. We update the decision matrix with the newly added Ability to Achieve metric, as seen in Table 8.2.

The updated matrix tells a different story than Table 8.1, which only leveraged ROI metrics. From this analysis we still determine that the Midwest is the best choice, but this time instead of the Southeast, we determine the Northeast would be the second-best choice. These two segments have the highest probability to achieve the expected outcome. The West and Canada have a neutral ability to achieve the expected outcome while we have less confidence in our ability to achieve the expected ROI in the Southeast. Notice that Canada is by far the most profitable segment and has a high ROI, but the probability metric makes us question how much money to allocate to this segment.

The prior example has a relative descriptive factor to describe probability. Let's look at an example of using a probability score. In Table 8.3, we work with our data scientist to build a propensity model that scores at a customer level to determine who is most likely to accept our offer. The score provides a probability to accept the offer based on demographic and financial credit information. We also have limited marketing dollars and want to apply them against the highest likelihood for success. In this case, we choose everyone above 65 percent.

Developing a probability factor for your decision matrix takes time and resources, but it can be a large factor in the success of your analytical solutions and monetization strategies.

Prospect Theory

Developed by Daniel Kahneman and Amos Tversky in 1979, prospect theory describes how people make decisions around economic risk. The theory puts forth that people do not interpret risk rationally in economic terms. Daniel Kahneman elaborates on several elements that comprise prospect theory in his book, Thinking, Fast and Slow. We cover three of these components: certainty effect, loss aversion, and diminishing sensitivity.

• Certainty Effect—People have a strong preference for certainty and are willing to forgo opportunities in order to achieve it. For example, a person may choose a 100 percent chance of winning $100 versus a 90 percent of winning $150. The certainty effect reinforces our advice to include a probability or confidence factor in your Decision Architecture, helping to lead an executive to a better decision.

• Loss Aversion—All things being equal, people will give higher weight to a perceived loss than the equivalent gain, even if the final outcome is the same. To elaborate, the utility gains from receiving $100 should be equal between one situation in which you are given $100 and another in which you are given $200, but then lose $100. In both scenarios the end results are $100, but the perceived loss of $100 in the second scenario is viewed less favorably than just being given $100 outright.

  While it may be tempting to show both the costs and benefits of a particular decision, our internal biases may influence our judgment if the costs are too high. The cost associated with a decision may be perceived as a loss. Loss aversion tells us that when we frame a decision matrix, be careful in displaying the outcomes that have gains and losses in order to avoid triggering loss-aversion bias.

• Diminishing Sensitivity—People tend to focus on relative differences rather than absolute differences. For example, the subjective perception is that the difference between $100,000 and $95,000 is much smaller than between $1,000 and $6,000. Both are $5,000, but we weigh the difference in the second scenario higher.

  When composing your decision matrix, the payoff amounts and how they are depicted could steer someone in the wrong direction. Add confidence factors and velocity metrics as needed to help overcome diminishing sensitivity.

Choice Architecture

Choice architecture helps improve decision making by presenting the options in a carefully structured process. This can be accomplished in a number of different ways, from the number of choices to the default choice provided to managers.

A great example of choice architecture is the Save More Tomorrow Plan, in which individuals are opted into employee retirement savings plans. In their article, “Leaders as Decision Architects,” John Beshers and Francesca Gino elaborate on the findings behind a default option with the retirement savings plan:

Listed here are three components of choice architecture that may help you with your monetization strategy:

1. Defaults—Consumers and users are more likely to choose a default option when one is provided. Defaults can help a person navigate to the correct choice. This phenomenon may be due to a number of reasons, one of which is that individuals interpret the architect of the solution as an authority and accept the presented option. Whatever the reason, consider setting defaults to steer an information user to a successful decision.

2. Reducing Choice Overload—When someone has too many choices it reduces their motivation to make a choice, which is called choice overload. For example, in her jam study, Sheena Iyengar offered a selection of two different booths for tasting jams. One booth had 24 options and the other had 6. Sixty percent of the customers were drawn to the booth with 24 options of jam while only 40 percent were drawn to the booth with 6 options. The findings on purchases is where this gets interesting; 30 percent of the people who sampled from the smaller assortment decided to purchase while only 3 percent of the larger booth made a purchase.

   It is better to present only a few choices or, in our case, decisions, to enable a better outcome. We can limit the number of decisions presented to a manager by providing a signal such as an alert or probability to help them quickly focus on issues that require attention rather than having to consider all options.

3. Avoiding Attribute Overload—When a product has too many attributes, it makes it difficult for consumers to evaluate options. For example, if a box of cereal has 20 different attributes that are displayed in the marketing message to the consumer, the customer will have a difficult time understanding how this cereal compares to other cereals in order to make an informed choice.

   This holds true when designing your monetization strategy. If you put in too many metrics for a person to evaluate in order to make a decision, you may cause analysis paralysis. We recommend prioritizing the metrics to just those that drive action to enable better decisions.

To read more on this subject, we recommend the book Nudge: Improving Decisions About Health, Wealth, and Happiness by Richard Thaler and Cass Sunstein.

Cognitive Bias

Cognitive bias is where an individual holds a view of a situation or object that is based on their subjective experiences, which may not be completely consistent with objective reality. Cognitive bias can play a big role in how we make decisions, which presents one of the most difficult challenges when composing your monetization strategy. This type of bias can impact many of the process steps within our methodology, interviews, working sessions, metrics, actions, decisions, and so on.

We are not suggesting that our learned experiences are not extremely helpful in guiding us to make effective decisions. However, these experiences should inform our decisions, not cloud them. We can often let prior experiences influence decisions to our detriment.

There is an entire field of study on this topic, so let's not go too deep, but consider that biases exist as you design your solution. Look to confirm a bias with facts or challenge the bias before making it a part of your analytical solution.

Here are some of the top cognitive biases we see when building our solutions:

1. Confirmation Bias—Confirmation bias is the inclination to look for information regarding a particular topic that confirms existing opinions, beliefs, and thoughts. If you are conducting an interview or working session, you may have to spend time priming the team with brainstorming activities related to thinking outside the box or keeping an open mind to assist with overcoming confirmation bias.
2. Recency Bias—This bias refers to the propensity to weigh the most recent information heavier and as more important than older data. This can often result in an information user not drilling into an issue to look deeper for causality while accepting surface-level information only.
3. Anchoring Bias—Opposite of recency bias, anchoring bias can be described as the inclination to anchor too heavily on the first piece of information you hear while putting less importance on subsequent data points. This is an important bias to recognize when working in sessions. For example, the order that the information is presented in will be important to determine the right Success Metrics and not just the first metrics that are considered.
4. Overconfidence—This bias can cause a person to overlook the facts of a situation that challenges their current understanding. This is probably most commonly found in experts who have been in a particular field for a long time. Experts may be overconfident in their understanding of the information, leading them to false conclusions that do not fully take into account changes in technology or market conditions.
5. Bandwagon Effect—The bandwagon effect occurs when a person adopts a belief or perspective because others have adopted a similar belief. This can be seen in workshops or group events as individuals with seniority can dictate the beliefs or decisions of the team. The bandwagon effect is one of the reasons we prefer to start with independent interviews to construct the decision architecture before team-based working sessions.
6. Clustering Illusion—The propensity to glean insights from random events is called the clustering illusion. We can often associate random events together due to proximity of the events. Our responsibility is to challenge this bias and prove it through data science. Data science can help us uncover relationships in the data to validate whether the clustering is real.
7. Outcome Bias—When a decision is based on the outcome of a prior event with little regard for how that event developed, outcome bias occurs. This can be especially true when trying to correlate marketplace activities with internal corporate strategies and initiatives, such as a big uptick in orders associated with a branding strategy. There may be some correlation between the two events, but you should be careful to consider other probable causes for the increase in orders such as a recent improvement in the economy.
8. Salience Bias—The propensity to focus on metrics that we are comfortable with versus a newer metric that we may not completely understand is known as salience bias. User training is vitally important to fight salience bias to ensure people adopt all available tools.
9. Zero-Risk Bias—People prefer certainty, even to their own detriment. Zero-risk bias is the inclination for reducing or removing a small risk to zero versus reducing another risk that may have a bigger impact. This can manifest itself in a decision to reduce risk in one manager's area at the expense of increased risk for the larger organization.
10. Empathy Bias—The propensity to underestimate the influence of one's feelings in the decision-making process. For example, a sales manager may have a winning personality, leading you to unknowingly overestimate that manager's actual performance.
11. Ostrich Effect—This bias is centered on a person's ability to ignore dangerous or bad data. To combat this bias, threshold signals make bad data difficult to ignore and can help an information user pay more attention to the issue.

Biases exist and it is our job to understand when they occur and architect a solution to minimize them. It is important to make sure that when you are driving out the decisions that fuel the Decision Architecture you apply the right amount of challenge to minimize the impact of cognitive bias on the outcome.

We covered a lot of ground in this chapter and provided you with many tools, or Lego pieces, that you can now utilize as you build your solutions. From Probability and Decision Matrixes to Choice Architecture and Cognitive Bias, working to create the right conditions for a quality decision is a multidisciplinary approach. The techniques in this chapter should help you structure the decision process to guide a person to the correct decision.