How to fine-tune your powers of prediction

I have a confession — I’m a sucker for predictions. If I come across (as I did some months ago) a headline like “Stock Market Crash of 2016: The countdown begins,” there’s no way that I’m not going to read it.

I know the statistics. I’ve read the books. I’ve written columns about it. Predictions are almost always wrong, and in the rare case they’re correct, luck is more likely the reason than skill. While I’ve trained myself to be more sanguine about market volatility, scary headlines, and risk in general, I’m as human as the next person. There is a part of me that will always crave the certainty that predictions promise.

Even the blatantly self-promotional “sky is falling” or apocalyptic narratives that flood my inbox are impossible to ignore. Based loosely on verifiable facts, they work because they play on our legitimate fears of what would happen if the financial world as we know it stopped working, and the wealth that took us so much time to accumulate simply disappeared.

In the back of our mind is the inkling that perhaps the writer has some insight or information we don’t possess. Fortunately we don’t have to dig too far before it becomes clear that the real driving force behind the “research” is income for the author.

That still leaves a small percentage of predictions that are generated from sources of real substance. Written by authors who are serious, experienced and respected, their intent is the open dissemination of important information and a reasoned discussion of its implications. Even then there is no guarantee.

Delayed crises

Some of you might remember Paul Ehrlich’s 1968 book “The Population Bomb.” Its prologue opened with these words: “The battle to feed all of humanity is over. In the 1970s hundreds of millions of people will starve to death in spite of any crash programs embarked upon now.”

Ehrlich was and remains a respected biologist and faculty member of Stanford University. His arguments were built on logical, well-reasoned science. No one could accuse him of drumming up fear just to sell his book. But Ehrlich’s prediction never actualized. Nearly 50 years later Ehrlich continues to warn about a pending problem — and he may yet be right. But his original prediction wasn’t that “at some point” we would run out of the ability to feed ourselves — it was that the crisis is imminent.

Jeremy Grantham is one of those rare people on Wall Street: an incredibly successful investor, creative thinker and writer with the courage to speak his mind in an industry that has turned saying nothing of substance into an art form. If you’re looking to hang your hat on a prediction, you’d be hard pressed to find one that is as clearly reasoned as his.

When Grantham writes that bubble territory for the S&P 500 starts at 2250, you know he’s not just throwing out a number. In the spring of 2015 he laid out his case for the potential catalysts to drive the current bull market into bubble territory but left some wiggle room for a “normal” decline of 10–20 percent, postponing the ultimate blowout until 2016 election. The market fell 12 percent in late August — so events appeared to be tracking his prediction perfectly. Grantham has been notably prescient at least twice before (prior to the Internet bubble, and prior to the financial crisis), but like every human being who has ever walked the planet, he’s not infallible.

Let’s say we are convinced that Grantham is onto something and the market is going to hell in a handbasket. We need to know how to represent that expectation in a rational investment plan. More importantly, if things don’t play out the way Grantham suggests, we need to have a built-in mechanism that regularly audits our perception of the world so we don’t end up chained to what may turn out to be an increasingly inaccurate prediction.

But we can’t use that mechanism until we overcome considerable confusion about the very nature of prediction: what it is and what it’s for. As long as I can remember, prediction tended to be this wild and perverse guessing game whose odds of success were not much better than the lottery; but it persisted because of our instinctive and persistent discomfort with uncertainty. Left in the shadows was a much more constructive way to understand and use prediction — one that had been around for over 200 years.

When someone makes a prediction, we don’t expect them to call a press conference every few days revising it little-by-little until it and the events that ultimately play out coincide. Prediction is all about confidently taking a bold stance and holding on to it until you’re proven right or wrong. Who would be interested in predictions if there weren’t any drama? As it turns out, just about anyone who is serious about anything.

Textbook case

In his book “Thinking Fast and Slow,” Daniel Kahneman relates a story that he regards as “one of the most instructive experiences of my professional life.” It relates directly to the challenge of prediction. During his time at Hebrew University he assembled a team of professors, graduate students and the dean (an expert in curriculum design) to design a course and write a textbook.

After a year of work, Kahneman asked each member of the team to write down how long they thought it would take to submit a finished draft of the textbook. The estimates ranged from 18 months to two and a half years.

Then he asked the dean how long he thought it would take. The dean said, based on his experience, there was a 40 percent likelihood that they would never even finish — and if they did finish it would take approximately seven years. Kahneman asked him how this group compared, in skills and resources to the others he knew about. The dean replied, “We’re below average, but not by much.” They ultimately completed the project — taking eight years. But by then the demand for the coursework had diminished and the textbook was never published.