Put Your Money Where Your Metrics Are

Table 13.1 shows common summary financial information for two hypothetical companies: Boom and Cruise. Income statement data from five years provide the basis for comparing the companies on several dimensions.

Hiding Problems in the Marketing Baggage?

The income statement for another example firm, Prestige Luggage, is depicted in Table 13.5. The company seems to be doing quite well. Unit and dollar sales are growing rapidly. Margins before marketing are stable and quite robust. Marketing spending and marketing-to-sales ratios are growing, but so is the bottom line. So what is not to like?

Smoking More, but Enjoying It Less?

Table 13.8 displays marketing metrics reported by a big tobacco company aimed at analyzing the trends in competition by lower-priced discount brands. A declining market size, stagnant company market share, and growing share of firm sales accounted for by discount brands combined to paint a baleful picture of the future. The firm was replacing premium sales with discount brand sales. To top it off, the advertising and promotion budgets had almost doubled. In the words of Erv Shames, former CEO of General Foods and president of Kraft, it would be easy to conclude that the marketers had “run out of ideas” and were resorting to the bluntest of instruments: price.

The picture looks much brighter, however, after examining the metrics in Table 13.9. It turns out that in the same five years during which discount brands had become more prominent, sales revenue and operating income had both grown by over 50%. The reason is clear: Prices had almost doubled, even though a large portion of these price increases had been “discounted back” through promotions. Overall, the net impact on the firm’s bottom line was positive.

Now you might be thinking that the messages in Table 13.9 are so obvious that no one would ever find the metrics in Table 13.8 to be as troubling as we made them out to be. In fact, our experience in teaching a case that contains all these metrics is that experienced marketers from all over the world tend to focus on the metrics in Table 13.8 and pay little or no attention to the additional metrics—even when they are given the same level of prominence.

The situation described by the two tables is a close approximation to the actual market conditions just before the now-famous “Marlboro Friday.” Top management took action because they were concerned that the series of price increases that led to the attractive financials in 1992 would not be sustainable because the higher premium prices gave competitive discount brands more latitude to cut prices. On what later became known as “Marlboro Friday” (April 2, 1993), Phillip Morris cut Marlboro prices by $0.40 a pack, reducing operating earnings by almost 40%. The stock price tumbled by 25%.

Note the contrast between this example and the preceding example. Prestige Luggage was increasing promotion expenditures to expand distribution. Prices were falling, while promotion, or sales on deal, were increasing—an ominous sign. Marlboro, on the other hand, was constantly raising the price and then discounting back—a very different strategy.

Marketing Dashboards

The presentation of metrics in the form of management dashboards has received a substantial amount of attention in the past several years. The basic notion seems to be that the manner of presenting complex data can influence management’s ability to recognize key patterns and trends. Would a dashboard, which provides a graphical depiction of information, make it easier for managers to pick up the ominous trends?

The metaphor of an automobile dashboard is appropriate because there are numerous metrics that could be used to measure a car’s operation. The dashboard is meant to provide a reduced set of the vital measures in a form that is easy for the operator to interpret and use. Unfortunately, although all automobiles have the same key metrics, it is not as universal across all businesses. The set of appropriate and critical measures may differ across businesses.

Figure 13.1 presents a dashboard for the Prestige Luggage example above of five critical measures over time. It reveals strong sales growth while maintaining margins even though selling less expensive items. Disturbingly, however, the returns for the retailer (GMROII) have fallen precipitously, and store inventories have grown. Sales per store have similarly dropped. The price premium that Prestige Luggage can command has fallen, and more of the company’s sales are on deal. This should provide a foreboding picture for the company and should raise concerns about the ability to maintain distribution.

Summary: Marketing Metrics + Financial Metrics = Deeper Insight

Dashboards, scorecards, and what we have termed x-rays are collections of marketing and financial metrics that management believes are important indicators of business health. Dashboards are designed to provide depth of marketing understanding concerning the business. Many specific metrics may be considered important—or even critical—in any given marketing context. We do not believe it is generally possible to provide unambiguous advice on which metrics are most important or which management decisions are contingent on the values and trends in certain metrics. These recommendations would have to be of the “if, then” form, such as “If relative share is greater than 1.0 and market growth is higher than change in GDP, then invest more in advertising.” Although such advice might be valuable under many circumstances, our aims were more modest—simply to provide a resource for marketers to achieve a deeper understanding of the diversity of metrics that exist.

Our examples, Boom versus Cruise, Prestige Luggage, and Big Tobacco, show how selected marketing metrics can allow deeper insights into the financial future of companies. In situations such as these, it is important that a full array of marketing and financial metrics inform the decision. Examining a complete set of x-rays does not necessarily make the decisions any easier. (The Big Tobacco example is debated by knowledgeable industry observers to this day!) However, it does help ensure a more comprehensive diagnosis.

The Gross Model

The Gross model, named after Irwin Gross, advises how much of the budget should be spent on creating (and testing) advertising copy. The number of alternative copies you should be willing to develop depends on the variability of the effectiveness of the advertisements. If some advertisements are highly effective, but most are ineffective, you will want to spend relatively heavily on developing and testing copy. The potential upside is high, and you want to develop many versions in order to get a great version. If, however, all advertisements perform relatively similarly, the difference between what you already have and what you can gain with further development is quite limited. In that case, it is better to spend less on developing new copy and spend more of the budget showing the currently best advertisement.

In the Gross model, the effectiveness of an advertising campaign (Z) is the amount spent on buying media (D) multiplied by the effectiveness of the best advertisement created (E):

Z = E * D

If we assume that advertising effectiveness is linear—that is, each piece of spending is equally effective—this is a simple model. Unfortunately, advertising often takes multiple views to gain any traction and eventually loses effectiveness; recall the S curve described in Chapter 10, “Advertising and Sponsorship Metrics.” This means E varies with the amount spent on media (D), making the model more difficult to use.

The amount spent on media (D) is the total budget (B) less the fixed costs of copy testing (C_F) and the total costs of each new advertisement. This is the number of advertisements (N) developed multiplied by the average cost to develop an advertisement (C) plus the marginal (extra) costs to test each advertisement (C_S):

D = B – C_F – N(C + C_S)

O’Connor and her colleagues¹ applied this formula using the historical distribution of advertising effectiveness and concluded that 20% to 30% of media budgets (B) should be spent on developing and testing, and the rest (D) should be spent on deployment. Of course, this rule of thumb may vary significantly depending on context.

Should You Test Another Advertisement?

The original Gross model was designed to help managers decide how many advertisements to create and test. The general conclusion was that firms tended to spend too little on creating and testing alternative versions (perhaps because this budget item is often called “non-working” media expense).

We will consider a slightly different question: Should the firm develop an additional advertising copy execution or use all of the remaining budget to air the current best copy?

Let B be the total budget to create, test, and deploy the best testing advertising. Let C be the cost to develop and test a new version of the advertisement—that is, the expenditure incurred when commissioning a new piece of copy. Because tests are never perfectly reliable or valid, we propose a “vaguely right” adjustment factor (A). The formula for the adjustment factor is Reliability * Validity (an approach similar to that proposed by Irwin Gross). As the reliability and validity of the tests approach 1, the adjustment factor approaches 1. As A gets higher—that is, whenever the validity and reliability of the test gets lower—each test is less useful at predicting real-world performance. The intuition is that with low reliability and validity, the test would need to indicate a higher probability of increased effectiveness before we would spend the money for the test. In our model, we use A = 2, which means the test would need to indicate an expected return two times (2X) the cost before we would deem it acceptable. This 2X would, for example, result from reliability of 70% and validity of 70%.

We can now estimate how much lift we need to expect to gain from a new version of an advertisement to make commissioning it worthwhile. This break-even lift is the cost of commissioning the new version as a percentage of the free budget multiplied by the adjustment factor. If we have a budget of $2.25 million, and the cost to develop and test each version is $40,000, then developing a new version costs 1.8% of the budget. If the adjustment factor is 2, we must expect to gain at least 2 * 1.8% = 3.6% performance lift to make commissioning a new version of the advertisement worthwhile (see Table 13.11). Of course, this approach means that each time we spend money for a copy test, the remaining budget is also reduced, and the next copy decision will represent a higher percentage of the budget and, therefore, require a higher percentage expected increase in sales to justify undertaking the test.

Will a new version of the advertisement give sufficient lift to make creating it worthwhile? The challenge in answering this question is that we don’t know any version’s effectiveness before it has been created and tested.

The expected value of the new version is the chance of getting a new version of a certain quality multiplied by the outcome when we get a new version of that quality. There are two broad outcomes. The new version is equal to or of worse quality than our best current advertisement. When this is the case, the new version has no value at all. When the new version is of higher quality, then lift is the additional value coming from the new version.

We make the assumption that each version of an advertisement has a quality score between 1 and 10, and a quality of 10 is twice as effective—in some way defined by the firm—as 5 and so on. For our example, we assume that there is a uniform distribution of quality for versions of the advertisement. If we have 10 levels of quality, 1–10, each is equally likely. (Note that one of the advantages of this model is that you can specify any distribution of advertisement quality that you wish; just change the distribution in Table 13.12.)

** Lift from New Version =IF(QNew>QCur, (QNew-QCur)/QCur, 0)

The lift from the new version is the quality of the new version minus quality of the current version divided by the quality of the current version. We know the quality of our current version—what we will deploy if we end testing—and so can create an expected lift from a new version. If this exceeds the expected lift needed to justify commissioning a new version (refer to Table 13.11), we should do so; otherwise, we should stop testing and deploy our current version. Table 13.12 shows us that with a current version quality 8, we should continue testing, but it is a very close call. (To read Table 13.12, note that the current advertisements scores an 8. Looking across shows us that a new version that scores 8 or less will not provide an effectiveness lift. A new advertisement that generates a score of 9 will improve lift by 12.5%.)

Whether you should continue creating versions depends on the quality of your current version. Table 13.13 shows the expected lift at each quality of current version. When you have only a low-quality current version, developing new copy has a huge expected value as you are likely to get copy that will substantially increase the effectiveness of your media spending. You should stop testing if you have a current version of quality 9; the expected benefits of continuing to commission versions are less than the cost of doing so. If you already have a version of quality 10, there is no possible benefit to further testing.

This model contains a number of assumptions—for example, that the version of an advertisement can be neatly scored, and the quality translates predictably into relative sales results. Furthermore, we assume that each new copy comes from the same distribution; this somewhat implies that the agency isn’t giving you its best ideas first. This need not be true. You might get progressively worse ideas each time you go to the agency. Despite the challenges in getting a perfect model, we think the method is most valuable as a general illustration of how to think about the value of information in a management decision context that includes uncertainty, financial consequences, and ability of imperfect data to inform a decision.

Further Reading

Ambler, Tim, Flora Kokkinaki, and Stefano Puntonni. (2004). “Assessing Marketing Performance: Reason for Metric Selection,” Journal of Marketing Management, 20, 475–498.

McGovern, Gail, David Court, John A. Quelch, and Blair Crawford. (2004). “Bringing Customers into the Boardroom,” Harvard Business Review, 82(11), 70–80, 148.

Meyer, C. (1994). “How the Right Measures Help Teams Excel,” Harvard Business Review, 72(3), 95.

O’Conner, Fina Colarelli, Thomas R. Willemain, and James MacLachlan. (1996). “The Value of Competition among Agencies in Developing Ad Campaigns: Revisiting Gross’s Model,” Journal of Advertising, 25(1), 51–62.