18. Value Quantification

In the beginning—the early days of the IT industry—the justification of new systems products was pretty straightforward. The systems we were installing then were usually replacements for manual clerical schemes. The labor saved was the value, and the development expense was the cost. The cost-benefit study came up with nice formulations like this:

To show we had a hardheaded respect for the time value of money, we expressed the various streams of costs and savings in terms of net present value (NPV). We could make easy statements of the form, “deciding to kick this project off now is the equivalent of adding a net present value of $1.3 million into the corporate coffers today.”

Sometimes, the justifications took slightly different forms:

TDM:    One of my early management responsibilities was to run a project that installed a centralized billing application in the French National Merchandise Mart at La Villette, in Paris. The new Mart was planned to replace the old one at Les Halles. At La Villette, all billing information was transmitted digitally. The manual form of the same function at Les Halles had utilized a network of pneumatic tubes to send chits and receipts and invoices whizzing around the floor under air pressure. The pneumatic tube network had been installed at Les Halles in time for the World Exposition of 1897. In 1897, there wasn’t much in the way of rubber products to make an airtight seal, so the tubes were all made of lead. When they built Les Halles, the price of lead was only a few centimes per kilogram. When we tore it down, the price of lead was more than seven francs per kilogram. And there was a lot of lead. The salvage value of the lead was enough to pay for the entire project, including all the hardware and software.

While the benefit side of the cost-benefit analysis was becoming ever softer, requirements for rigor and precision on the cost side were increasing. So, it became common to see a new project justification like this:

Cost = $6,235,812.55
   Benefit = “We gotta have it.”

When projects have precisely limited costs and only vaguely stated benefits, development personnel are held accountable for costs and nobody is held accountable for benefit realization. The project is then inclined to shed functionality willy-nilly in order to meet cost targets. Since nobody bothered to state where the greatest benefits lay, there is no valid basis for shedding one function as opposed to another. The most common result is poor benefit realization and pointing fingers on all sides.

Precisely stated costs and vaguely hinted-at benefits make a travesty of cost-benefit analysis. More importantly, they also make sensible risk management impossible. When risks are considered in isolation, there is no way to justify any given amount of risk. The result is that only the most risk-averse approach seems to make sense.

This is all leading toward an inevitable principle:

When benefit cannot be stated more precisely than “We gotta have it,” then the cost specification should be “It’s gonna be expensive.” If the costs are specified with a clearly implemented risk diagram, the benefits have to be stated in a similar form (for more about this, see Chapters 21 to 23).

Similarly, the stakeholders have to be held accountable for benefit-predicted and benefit-realized. The precision or imprecision of these benefit quantifications has to be more or less equivalent to the precision or imprecision of the costs.

Other reasons companies don’t do careful benefit predictions and benefit-realization assessments include the following:

• The system is too small for us to bother.

• There is no choice about whether or not to build this system.

• The system is required by a regulatory authority.

• The benefit depends entirely on meeting the market window.

• The system is a replacement for an existing system.

• The request comes down from on high.

• The benefit is too uncertain to quantify precisely.

• The stakeholder has said, “Trust me, it’s worth doing.”

• The benefit numbers wouldn’t be credible anyway.

On this last point, our colleague Steve McMenamin, at the time a vice president of Edison International, offered this observation:

There is a class of purported savings that I term “general productivity.” These are of the form: “If we implement the new data-mining system, each employee will save at least two minutes per day, which adds up to annual savings across our whole organization of forty-two kajillion dollars.” It’s not that there isn’t a grain of potential truth in such claims. But they’re such a reliable hiding place for stupid projects and the consultants who propose them that claims of “general productivity” benefits receive withering and usually well-deserved scorn. I typically discount them by at least one-hundred percent.

The gripe here is about having tiny benefits distributed widely. When productivity benefits are huge and concentrated, the justification can be much more compelling.

Not included in our list of reasons companies don’t do careful benefit predictions and benefit-realization assessments is one that is often applicable but never mentioned: The benefits are minuscule or nonexistent. As a general rule of thumb, when benefits are not quantified at all, assume there aren’t any.

An aggressive risk-taking posture has to be steered by benefit. How much risk you’re willing to take has to be a function of how much benefit there is to capture.

TRL:    In the 1990’s, many of my clients got fixated on improving the wrong process. They were all hung up on the how-projects-are-built process. The one that really matters is the process that determines which projects are worth doing. Ironically, in some of the most process-aware companies I know, there is no defined process for project initiation—it’s all done by fiat.

• Stakeholders declare expected benefits at the same time that developers declare expected costs and schedules, and to equivalent precision.

• Stakeholders express the uncertainty in their benefit expectation in the same way that developers indicate the uncertainty in their cost and schedule calculations (see Chapter 21 for details).

• Stakeholders assess the relative value of system components in order to provide a basis for version selection and to perform sensible sensitivity analysis and incremental cost-benefit analysis (details in Chapter 22).

• Management justifies the project based on a careful comparison of benefits and costs and their attendant uncertainties (details in Chapter 23).

• Management assesses benefit-realized after the fact and provides the assessment as input into the postmortem process.

This approach is broadly what Barry Boehm calls Value-Based Software Engineering. Boehm comments on the need for value-basing:

Software engineering is fundamentally a contact sport. In software engineering as in rugby, no amount of theory about how to succeed in a scrum is going to come close to real experience in the middle of a few scrums.

Further, the theory that most current students get covers maybe 15% of the activities they encounter in practice. Most of it is based on a model of software engineering as a set-piece job of deriving and verifying code from a static set of requirements. This was a good model in the 1970’s . . . but it’s way out of date now. In the 1970’s, software was a small part of most systems, and the stability of the requirements meant that you could often “separate concerns” and just attack the problem of deriving code from software requirements in isolation. But all of that has changed now. Software is critically bound to a system’s value-added, its flexibility is the key to adaptation to change, and software engineering is less and less “all about programming.”¹

¹ Barry W. Boehm, “Software Engineering Is a Value-Based Contact Sport,” IEEE Software (Sept.–Oct. 2002), pp. 95–97. Reprinted by permission.

In this view, the focus is as much on the value to be realized as on the “set piece” of software production.