Vice Admiral Robert FitzRoy is a man whom most people will not have heard of, but should have—for at least two reasons.

Readers would likely fail to name FitzRoy as the captain of HMS Beagle, the ship on which Charles Darwin sailed when Darwin was formulating his thinking on evolution through natural selection, thoughts that eventually saw the light of day in The Origin of Species.

What is even less well known is that FitzRoy was the man who founded what was later to become the British Meteorological Office. Furthermore, he was the one to coin the term forecast to describe his pioneering work. In The Weather Book, published in 1863, he wrote: “[P]rophesies or predications they are not; the term “forecast” is strictly applicable to such an opinion as is the result of a scientific combination and calculation.”

A century and a half later, the Met Office is still around and still involved in “scientific combination and calculation.” The intervening years have seen enormous advances in the understanding of the physics of weather systems, in the speed, quality, and quantity of data collection, in mathematical techniques, and in computational power. Today, organizations like the Met Office own some of the most powerful computers on the planet. As a result, weather forecasts are significantly more accurate than they were even 10 years ago.

Despite these advances, it is still not possible to forecast the weather with any degree of confidence more than a week or so into the future—and it almost certainly never will be. This is because there are practical limits to what it is possible to predict using any approach known to man.

Our everyday experience of weather forecasts serves as a salutary lesson to those working in the messy and complex world of business who might be tempted to believe that the path to better forecasting lies in using ever more sophisticated mathematics. However, despite what we know about our ability to predict the weather, people with a naïve faith in the power of mathematics are not hard to find. This is good news for some software vendors who make a handsome living from selling exotic black-box forecasting solutions to clients who want to believe that a fancy system will somehow make their forecasting problems disappear.

Happily, the editors of this book do not share any of these shortcomings. This is not because they lack technical expertise—far from it—nor is it because of a lack of faith in the value of forecasting to business. It is because they have the intellectual self-confidence to recognize the limits as well as the value of mathematical computation, the humility to be open to new ideas, and the honesty to let results be the judge of what is right and good. I respect and admire these qualities, so I was happy to write this foreword.

But if more math is not the “silver bullet” for forecasting, what is?

I cannot improve on the analysis advanced by David Orrell in his excellent book, The Future of Everything. He argues:

• Mathematical models interpret the world in simple mechanical terms, which have limited applicability in the context of complex systems such as living systems or systems that contain living systems, such as economies and organizations.
• Such living systems have properties that elude prediction. This is not just because such systems are complex; it is because they adapt and evolve. Their very nature involves making the future different from the past, which limits our ability to forecast the future by extrapolating from what has gone before.
• Forecasting has a large psychological component. Human beings are not automata; we can be rational, but we are also passionate, intuitive, and impulsive, and the way our brains are wired makes our judgment prone to bias and hopeless at understanding probability. This is compounded by the fact that, in organizations, forecasts are often embedded in a political process where many stakeholders—such as those in sales, finance, and general management—have vested interests that can skew the outcome.
• Some predictions (forecasts) are still possible. The future is never the same as the past, but neither does it completely differ. So approaches that involve mathematical modeling based on what has gone before are an essential part of the forecasting process, not least because our brains need their help to deal with the complexity of the world.

Orrell concludes that we fall short of what is possible—and to get better, we need to change our approach to making predictions. His prescription involves a more eclectic approach, using multiple perspectives rather than having blind faith in a single algorithm. We should draw on different mathematical methodologies and supplement them with judgment and intuition.

This doesn’t mean abandoning rigor. We should aim to develop a deeper understanding of the mechanics of the systems we are forecasting, rather than treating them as a black box. We need to improve by testing our predictions against reality and learning from what our errors are telling us about the shortcomings of our methods. And forecasting should be embedded in a properly specified business process, run by appropriately trained and equipped professionals.

As practitioners, we should never lose sight of the fact that forecasting is only of value if it helps us deal with the real world. This means that we need to be able to explain and convince our colleagues, recognizing that not everyone will share our knowledge or perspective on the world or our motivation to expose the objective “truth.” It also means that we need to be able to balance the aesthetic pleasure we derive from an elegant piece of mathematics or a beautifully designed process with the usefulness of the results and the degree of effort required to produce them.

I believe that forecasting in business should be regarded as a craft. Good craftspeople understand the materials they are working with and know that their work will only be as good as the tools they use. But they understand equally that real skill comes from knowing how and when to use those tools. So we need craftspeople who are eclectic but rigorous, professional, and pragmatic.

Acquiring such knowledge from personal experience can take a lifetime, which is longer than most of us are prepared to give. What we can learn from others is worth a high price.

I don’t know of a better source of forecasting craft than this book—and I commend it to you.

Steve Morlidge

CatchBull Ltd.

London, UK