One of the most fascinating things about markets is the sheer volume of data they generate. Every day, millions of data points get created. The vast majority of this amounts to little more than noise. This endless stream of information leads thousands of us everyday on a hunt for meaningful signal amid the cacophony.
Most of the time, we are unsuccessful. We can be tricked into seeing significance where none exists. Our pattern recognition engines are fooled by what turns out to be mere randomness (there’s a good idea for a book in that). There is a meaningful needle buried somewhere in the haystack of numbers, but we underestimate both how hard it is to identify, and just how huge that stack is.
As an example, let’s take a number we are all familiar with: Average market returns. Depending upon the period of the data set under discussion, we usually envision a long-term annualized compound return of 8 percent to 10 percent.
Have a look at the chart below:
Source: Dimensional Funds
It shows the distribution of returns of the Standard & Poor’s 500 Index going back to 1926. During the course of the 89 years covered by the chart, we never had a single year when the annualized compound return was simply the average! This means that if you are expecting your returns to fall somewhere around 8 percent to 10 percent in a given year, you have — at least so far — been disappointed.
The market has, on occasion, been in the neighborhood of the median annual return of 14.3 percent, or the average annual return of 12.1 percent. But the occurrence is infrequent — less than 1 out of 10 years.
Average, as it turns out, is surprisingly rare. Radical standard deviations year-to-year only even out over long periods of time. In any given year, markets are much more of a roller coaster than the averages might suggest.
Some of the reason for this is simple statistics: We are working with a relatively small data set of less than 100 years. If you really want to look at a robust Gaussian distribution, we should probably revisit this conversation in April 3015. By the time we have accumulated more than 1,000 years of market data, almost all of the randomness should be eliminated.
We can slice and dice the numbers in other ways. Indeed, were we to look at rolling 12-month periods instead of calendar years, our data set expands from 89 data points to more than 1,000, the anomaly disappears.
Other mathematical oddities are present. Why on so many occasions — nine times — were annual returns 35 percent or higher? Why does the distribution of returns not look more like the usual smooth bell curve?
The returns above reflect what markets generate. Once we begin looking at actual investor returns, the data changes even more radically. The simple reason is human nature, and the many behavioral foibles that seem to trip up investors.
The average investor realizes returns of about 3.7 percent. That’s far below any of the market returns discussed above. This underperformance occurs regardless of the investment vehicle, whether it’s mutual funds, hedge funds or just plain old stocks. The reason investors tend to underperform is that they chase alpha (above-market returns) and fail, rather than aim for beta (market-matching returns) and succeed.
None of this is a great mystery. It is well known that most folks are doing something wrong — very, very wrong — with their investment dollars. We know that humans are irrational, are subject to wild emotional swings, and suffer from all manner of cognitive errors. It isn’t your fault; that’s just the way you are built. And your nature is the underlying cause of that poor performance. Recognizing this should help investors to make better, less emotional decisions.
Then again, that’s probably my emotional, irrational optimism speaking. Average returns are exceedingly rare — and for a good reason.