Undiversified portfolios are at a high risk of underperforming, Bessembinder says.

Here’s an eye-opener: From 1926 through 2015, only 4% of listed stocks were responsible for the overall net gain in the U.S. stock market. The other 96% collectively matched one-month Treasury bills over their lifetimes.

That’s what a working paper, “Do Stocks Outperform Treasury Bills?”, by finance professor Hendrik Bessembinder of Arizona State University, has found and discusses in an interview with ThinkAdvisor. He studied data on nearly 26,000 stocks going back to 1926.

The professor’s research helps explain why active managers most often underperform benchmarks. Since actively managed portfolios are typically poorly diversified, the research emphasizes the importance of diversification.

Bessembinder’s study suggests that active management’s underperformance can typically be anticipated as a result of undiversified portfolios — “even in the absence of costs, fees or perverse skill,” he writes.

The phenomenon is a consequence of “positive skewness on the cross-sectional distribution of stock returns.” Skewness is present in undiversified portfolios but not in fully diversified ones. Bessembinder explains this lack of symmetry, in succinct layman’s terms, in the interview.

If you’re surprised that the positive performance of the market is attributable to outsize returns of only a small number of “powerhouse” stocks, so was Bessembinder. He’d always assumed that over time, most equities would deliver a positive premium compared to Treasury bills, he says.

He got the idea for the study while working on another project for which he was analyzing data from Chicago University’s Center for Research in Security Prices (CRSP). Finding the monthly data to be negative, he began digging and then dug some more, which led to his paper.

Bessembinder has published more than 35 academic articles on the financial markets and consults to firms including Barclays Global Investors and Goldman Sachs. He has also consulted to the Financial Industry Regulatory Authority on transparency in bond markets, the New York Stock Exchange on measuring trading costs, the Securities and Exchange Commission on order routing and the Justice Department on stock market collusion.

ThinkAdvisor recently interviewed Bessembinder, on the phone from Seattle, where he conducts research every summer at the University of Washington. In our conversation, he does not neglect to opine on why investing in stocks still beats playing the lottery. Here are excerpts from the interview:

THINKADVISOR: What are the chief takeaways from your study?

HENDRIK BESSEMBINDER: The bad news for stock pickers is that there’s worse than a 50%-50% chance that the stocks will underperform. The good news is that if, by luck or wisdom, you’re able to identify those relatively few big winning, home-run stocks, there’s tremendous potential return. There is an enormous possible upside to active management if one can pick those super stocks in advance.

Net-net, what does that mean to financial advisors?

If you pick relatively few stocks as opposed to being broadly diversified, there isn’t a 50%-50% chance as to whether they’ll over- or underperform the market. There’s a greater than 50% chance they’ll underperform.

So is a key implication from your study that most advisors should build diversified portfolios?

For a lot of advisors, that goes against their core mission. To say that active managers should broadly diversify would go against the grain for them because if you broadly diversify, you’re essentially an indexer. The more diversified you get, the more you start looking like an indexer as opposed to being a true active manager. And if you’re an indexer, have you justified charging a sizable fee?

What does your research add to what has already been documented about active management’s performance?

We already know that most active strategies underperform, particularly over longer time periods. In addition to the explanations that are in the literature, like management fees and trading costs, the point that comes out of my paper is that the skewness of returns contributes to more than 50% of active strategies underperforming.

Is that mainly due to lack of portfolio diversification?

Right. The skewness issue is present in undiversified portfolios.

OK. Now, please define “skewness.”

In a positively skewed distribution, which is what I’m documenting, the median is less than the mean: Most of the individual returns are less than the mean, and a few individual returns are much greater than the mean. So most individual stocks are going to underperform.

Do the big home-run stocks have any common characteristics?

That relates to the question: Can we identify these stocks in advance? All I can say is that I don’t have any good insights on how to do that. Of course, the key question for active managers is whether they can convince their investors that they can.

What are some examples of the super stocks that you found?

Many were the big technology stocks, like Apple, Facebook, Microsoft. There were some old-school industrial stocks too, such as ExxonMobil, which is No. 1 in lifetime wealth creation. But they’ve been in the database since the 1920s. The other stocks of course have been there for much shorter periods of time — yet they’re high on the list.

You looked at returns as of July 26, 1928. That was a year before the start of the Great Depression.  How did the Depression and the Wall Street Crash of 1929 impact the returns you studied?

Of course the Depression weeded out some stocks. It’s in the database, and it’s relevant. But the phenomenon of the majority of stocks underperforming Treasury bills is concentrated in stocks that have joined the database since the 1970s.

Why is that?

It seems likely that we had a change in the type of company that was brought to market around that time. They were probably bringing younger and less prudent companies to market as compared to the stocks brought in earlier decades.

Do you agree with the reporter, writing in The New York Times, that interpreted your findings to mean that “individual stocks resemble lottery tickets”?

He was focusing on the small possibility of very large returns. The danger in that analogy is that lottery tickets lose money on average. The stock market does not lose money on average. So it’s important to make that distinction. Stocks are a much better investment than the lottery.

Certainly most people don’t create money by buying a lottery ticket.

Right. Buying lottery tickets is keeping the dream alive. But if you’re reasonably diversified in the market for a time, you do make money. You’re better off being a stock picker and seeing if you can pick the next Amazon. The problem is that you have to wait 20 years to see if you have.

What do you invest in?

I’m completely an indexer and largely buy-and-hold. I have equity index funds and low-cost bond funds. So I’m personally not a stock picker.

— Related on ThinkAdvisor: