APPENDIX C
An Introduction to Decision Trees
In this appendix, Robert Dees and Ken Gilliam briefly describe some of the fundamental concepts of decision analysis and move to a discussion about one of the favorite structural models of decision analysts: the decision tree.
Decisions and Outcomes
To think about decision analysis, we must begin with the decision. A decision, as already stated, is an irrevocable allocation of resources. In the project manager’s world, this means that the decision is not made until contracts are signed or money changes hands. We know that we have made a decision when we cannot go back, or at least that we cannot go back without incurring a penalty. Going back would be another decision, and in this case we have at least decided to commit resources up to the amount of the penalty. This penalty could be time, money, public opinion, or any other resource.
On the flip side, when we do nothing or wait to decide, we are choosing to allocate resources in a particular way right now, and we might incur an opportunity cost. We must decide about not only what to do but also when to do it. This definition of a decision also implies that a decision is more than just thought; a decision is an action.
We would like our decision to be characterized by logical thought, but it isn’t a decision until action takes place. Consider a person who says that he or she is on a diet but routinely visits the pantry for junk food; the dieter hasn’t truly decided on the diet until the actions performed reflect his or her spoken words. Along the same lines, a project manager hasn’t decided on a project management plan, or anything else, until it is implemented.
The most important, and still most commonly misunderstood, distinction in decision analysis is that between a decision and an outcome (Howard 2007). We have already defined a decision; we say that an outcome is a future state of the world, and that a good outcome is one that we prize relative to other possibilities (Howard 1988), as shown in Figure C-1.
FIGURE C-1: Decision and Outcome Relationships
When we make a good decision, we expect to have a good outcome but might experience an unlucky bad outcome. If we play the roulette game with an 80 percent chance of winning, we can still get unlucky and lose. When we make a bad decision, we expect to have a bad outcome, but we might get lucky and have a good outcome. As decision makers, we are unable to eliminate all uncertainty in the world, and we face the possibility of bad outcomes after making good decisions.
Decision Trees—“Hammer V-12 Roadster” Case
Confusing the tools of decision analysis with decision analysis itself has contributed to the loss of precision. Because uncertainty is at the heart of most perplexing decision problems, decision analysts frequently use specialized tools, such as decision tree techniques, to evaluate uncertain situations. Unfortunately, many people, some of them educators, have confused decision analysis with decision trees. This is like confusing surgery with the scalpel. Although decision tree techniques are extremely useful in solving problems where uncertainty is critical, in a real decision analysis, most of the effort and creativity is focused on finding and formulating the correct problem and on interpreting the results rather than on performing computations. (Howard and Matheson 2004)
Decision trees are only one tool; in Chapter 4, we mentioned other tools, including decision matrices, decision hierarchies, influence diagrams, spreadsheets, tornado diagrams, risk profiles, and sensitivity analyses. In this section, we proceed through a systematic development of necessary distinctions that must be understood in order to effectively use decision trees. We will use a hypothetical case about a product development decision to demonstrate these distinctions.
You are currently a lead project manager working for Startup Motors. Startup Motors makes performance vehicles and unveiled a new model called the “Hammer” on last year’s auto show circuit. The Hammer is a V-12 roadster designed to compete with the likes of the Dodge Viper and Chevrolet Corvette ZR-1. The estimated selling price is $85K to $95K. Based on the buzz at the auto shows, Startup Motors has decided to continue the development of the Hammer into a production model.
You have been hired as the lead project manager for Hammer development. Up to this point, Startup Motors has spent $30M in development of the Hammer, and you estimate that it will take an additional $80M to get the Hammer to market. In the past, 80 percent of vehicles introduced in this segment of the market have been a “Success.” If the Hammer is successful, then you believe that Startup Motors will realize a $120M profit over the current product planning horizon. If the Hammer experiences “Failure,” then Startup Motors will realize only a $10M profit over the same time period.
You, along with the top-level management of Startup Motors, hire an outside market research company to assess whether the Hammer will be a “Success.” In the past, this market research company’s predictions have been 90 percent accurate in this segment of the market.
Possibilities
To begin, we have said that the Hammer can turn out to be either a “Success” or a “Failure.” This distinction has two degrees, or levels, that we could realize. We easily could have defined more degrees; we could have rated the level of success on a scale from 1 to 10. We have kept it to two degrees for simplicity in our first example. “Success” and “Failure” are mutually exclusive, meaning that they cannot both happen at the same time. It makes sense that only one of our degrees of any distinction could happen. “Success” and “Failure” are also collectively exhaustive, meaning that there are no additional possibilities. Because “Success” and “Failure” are mutually exclusive and collectively exhaustive, exactly one of the two degrees on this distinction will happen. We represent this distinction in a possibility tree (see Figure C-2).
FIGURE C-2: Initial Possibility Tree
As we identify other relevant distinctions, we can add more branches to our tree. Using our Hammer example, we now decide that the marketing firm will conduct a market test and predict whether the Hammer will be a “Test Success” or a “Test Failure.” We see that there are now four possibilities that could happen. It is important to note that we could experience “Success” even if the Hammer is a “Test Failure.” Additionally, the Hammer could be a “Failure” even if we are told that it is a “Test Success.”
The order of the possibility tree (Figure C-3) shows that we are first going to think about whether the Hammer is a “Success,” and then whether the marketing firm reports a “Test Success.”
FIGURE C-3: Possibility Tree
This is the order of the information that we have at this point. As we progress, we would like to think about the results of the market test before we think about success of the Hammer because this is the order that the events will happen.
Outcomes and Value
We have said that outcomes are future states of the world. As of now, we have four possible outcomes in our Hammer example. We need to assess the value of each one of these future states. In this case, we are told that if the Hammer is a “Success,” then we will experience a profit of $120M. If the Hammer is a “Failure,” then we will experience a profit of only $10M.
Value is not always expressed in dollars; project managers continually care about cost, performance, and schedule. None of these are valued in the same units, and currently we are considering only one measure of value—dollars. Multiple objective decision analysis (MODA) can be used to assign value to our joint possibilities when we face a decision for which we have multiple and competing objectives (Parnell and Driscoll 2008). One of the attractive features of MODA is that it helps us to get everything into the same units so that we can compare apples to apples. For our current discussion of the Hammer, we make the assumption that our only value is money. We now move to include value in our possibility tree (Figure C-4).
FIGURE C-4: Possibility Tree with Value (the order we have information)
We have recorded the value of “Success” and “Failure” along the branches of the tree. We then add Value to the right through our tree to arrive at the Value associated with our joint possibilities. We note that our profit, in this case, doesn’t change based on what the marketing company says about the Hammer. We can record Value in the tree like this for problems with a single objective; we can add the Value to the right to come up with the Value of each possibility. Alternately, we can assign Value to each of our joint possibilities (prospects).
Probabilities
We now add our probabilities, or beliefs, to the possibility tree (Figure C-5). Notice that the probabilities sum to 1.0 on any individual branch of the tree; this is because our distinctions are mutually exclusive and collectively exhaustive. As mentioned in the Hammer case, the chance of success in this market segment is 80 percent; this is recorded as a probability of 0.80 in the tree. Additionally, we said that the marketing company has been 75 percent accurate at predicting “Success” and 25 percent accurate at predicting “Failure” in this segment. We sum value to the right; we multiply probability to the right. We see that the probability of “Success” and “Test Success” happening is (0.80 × 0.75) = 0.60. The probabilities of our four joint possibilities also sum to 1.0.
FIGURE C-5: Probability Tree with Value
In the information we have, the probability of “Test Success” is conditioned on “Success.” This works, but thus far we have talked about the information in the order that we have it, and we want to get the information in the order that the events will happen. Figure C-6 explains how to get the information in the order that the events will happen.
FIGURE C-6: Flipping the Probability Tree
This figure shows a process that we call “tree flipping.” We changed the notation for our joint possibilities. S indicates “Success,” and S’ indicates “Failure.” T indicates “Test Success,” and T’ indicates “Test Failure.” The tree on the left is identical to the previous figure, and we used the following steps to flip the tree:
We first drew the tree to the right in order to depict the information in the order that the events will happen. Hammer “Test Success” is now the marginal possibility, and Hammer “Success” is the conditional possibility. The joint possibilities (prospects) are the same as they were previously. Note that ST = TS; they are the same prospect.
We then use probability theory to compute the probabilities of “Test Success” and “Test Failure,” as well as the conditional probabilities of “Success given Test Success” and “Failure given Test Success.” (See Buede 2000, Howard and Abbas 2007, or Schuyler 2001 for more information on how this is done.)
Figure C-7 shows the final version of our probability tree. This tree shows the information in the order that the events will actually occur.
FIGURE C-7: Probability Tree with Value
The most counter-intuitive result that we have found thus far is that when the 75 percent accurate marketing company indicates that the Hammer is a “Test Failure,” we still have a 57.1 percent chance of “Success.” This is a well-known example based on the law of conditional probability, and it is used extensively in the field of medical testing. Right now, we have an 80 percent chance of “Success”; this is known as the prior probability of “Success.” If we get back “Test Success,” then we have a 92.3 percent chance for “Success”; this is called the posterior probability of “Success.” We have gained some information from the marketing test, and we will later talk about how much the test is worth to us.
Decisions and Decision Trees
We are now at a point where we can represent our decisions in the Hammer tree. We will insert our decisions in the order that they happen. Decision trees inherently grow in the order that things will happen, and we advocate modeling the decision this way when attempting to build a new tree. We use circles to represent chance nodes in the tree; we use squares to represent decisions. Additionally, we use triangles to denote terminal nodes (Figure C-8).
FIGURE C-8: Decision Tree
After we get the results of the marketing test, we will make a decision about whether to continue development of the Hammer. Notice that we have inserted the future development cost of $80M on the decision branches. We solve the decision tree by using the following steps:
Sum value from left to right. In our example, if we decide to continue development and the Hammer is successful, we will realize an outcome of -$80M + $120M = $40M. If we continue development and the Hammer is a failure, we will realize an outcome of -$80M + $10M = -$70M. If we decide to abandon the Hammer, we will realize an outcome of $0M from where we are now. We said that we have already spent $30M on the Hammer, but this is a sunk cost and should not be considered in future decisions.
Fold the tree. When we fold the tree, we (1) calculate the expected value of chance nodes and (2) choose the better alternative when we encounter a decision node (Clemen and Reilly 2001). Expected value is given by:
where pi is the probability of event i and Vi is the value of event i. An expected value is what we would expect to get as a long-run average if we were to make the same decision over and over again. When the chance node is at the end of the tree, we use the terminal value in our calculations. We illustrate these steps for our Hammer case:
We first calculate the expected value of our chance nodes relating to Hammer success. If the Hammer is a “Test Success” and we decide to “Develop” it, the expected value of our chance node is (0.923 × $40M) + (0.077 × -$70M) = $31.54M. We complete the same operation for our other chance node with different probabilities and have the resulting tree (Figure C-9).
FIGURE C-9: First Fold of Decision Tree
We choose the better alternative when faced with the development decision. In the “Test Success” case, we see that the expected value of the decision to continue development is greater than the certain terminal value of $0M if we choose to abandon the Hammer. We also notice that we should abandon the Hammer if the marketing company says “Test Failure,” because the expected value of deciding to develop is $7.19M less than that of abandoning the Hammer. This is indicated by “TRUE” or “FALSE” on our tree, and we can prune the tree back as shown in Figure C-10.
FIGURE C-10: Second Fold of Decision Tree
Finally, we might take heart in our current situation at Startup Motors; we have an expected value of $20.5M for our current situation. An expected value does not guarantee what we will get as the situation unfolds. From our initial tree, we can see that our only possible outcomes are -$70M, $0M, and $40M. As stated earlier, an expected value is what we would expect to get as a long-run average if we were to make the same decision over and over again. Once again, we choose based on expected value only when we agree to be risk-neutral over our prospects.