Portfolio > Mutual Funds

Morningstar Slams Study Questioning Its Rating System

Your article was successfully shared with the contacts you provided.

Morningstar logoMorningstar is embroiled in a dispute with two professors at Pace University over the effectiveness of its stewardship grade ratings system. Morningstar developed stewardship grades in order to evaluate the mutual funds it tracks by corporate culture and responsibility, in addition to its performance-based star rating system, believing stewardship has predictive value of future returns.

Aron Gottesman and Matthew Morey, finance professors at Pace University’s Lubin School of Business, released a study, “Mutual Fund Corporate Culture and Performance,” on Wednesday that tested the theory. Gottesman and Morey examined the ability of corporate culture ratings to predict risk-adjusted performance of domestic equity funds from 2005 to 2010.

“We find there is little significant evidence that corporate culture predicts better fund performance,” the researchers write. “Indeed, we find that no individual component of the Morningstar stewardship rating including board quality, fees, manager incentives and regulatory issues is able to consistently predict fund performance.

“Arguably the most important component of the Morningstar stewardship rating is fund corporate culture as it sets the tone for the entire operation of the fund,” they add. “Indeed, the culture of the fund conveys how well employees are treated, how the fund treats its own investors, and how well the fund communicates with its shareholders.”

As the authors note, in the wake of the late-trading and market-timing scandals in 2003, there has been a great deal of interest in the governance of mutual fund companies, the reason Morningstar developed the grades.

Unlike the star ratings, which focus only on past fund performance, the stewardship ratings examine five governance factors of the fund company itself: board quality, corporate culture, fees, manager incentives and regulatory issues. The stewardship ratings essentially allow an investor to determine how well the fund company is taking care of its fiduciary responsibilities, Gottesman and Morey say.

Morningstar reacted strongly to the Gottesman-Morey study in a detailed response, claiming it had “serious flaws.”

“First, we’re skeptical of any study that finds that fund expenses have no predictive value,” Morningstar said. “Expenses have been shown time and again to be predictive of performance. We think it’s also important to point out that Gottesman and Morey did find that a high culture score was strongly associated with low fees, long manager tenure, low turnover and fund survival. These are all very positive qualities in a mutual fund.”

Second, the company says the Gottesman-Morey study only examined domestic equity funds, which represent about half of Morningstar’s stewardship grade coverage, over a single five-year time period.

“While we generally wouldn’t recommend drawing conclusions from such a short period, it’s worth pointing out that the study did find predictive power over that full five-year period, just not the shorter periods,” Morningstar said. “Given the long-term nature of stewardship this makes sense. We wouldn’t necessarily expect the predictive power of corporate culture to show up in the one- and three-year periods because culture affects things such as how good the next manager is when the current one departs, long-term fee policies (see Vanguard Admiral), not allowing a fund to get too big, etc.”

Third, Gottesman-Morey’s method for addressing survivorship bias actually inflates the performance of funds with poor culture grades, Morningstar says. They replace dead funds’ track records after the death date with the performance of other funds with the same category, culture grade, expenses and turnover, but not the same performance history.

“I think it’s safe to say that most funds that were merged or liquidated in the periods they study did so because of poor performance,” the Morningstar statement said. “As a result, any surviving funds in the lowest culture group would have had much better performance than the ones that didn’t survive. Additionally, our stewardship coverage is biased toward larger, better-performing funds to begin with, so selecting any fund with stewardship coverage biases performance upward.”

The statement continued: “Fourth, one of Gottesman-Morey’s primary measures of ‘success’ is the Sharpe ratio, which does not take a fund’s category into account. This will muddle the results. For example, the differences in Sharpe ratio between large value funds and small growth funds are so big that they overshadow the differences between A Stewards and C Stewards.

“Finally, we conducted our own study last year that found that culture grades are predictive of fund performance. We examined how funds have performed since Morningstar first issued its stewardship grades in 2004 and again after we revised our Stewardship methodology in 2007. We concluded that funds with high stewardship grades (those receiving grades of ‘A’ or ‘B’) are very likely to survive in the long term, and more likely to provide competitive risk-adjusted returns in the ensuing period.”