While the ETF explosion has caught the attention of investors and their financial advisors, these popular index funds have also led a number of research shops to offer their own interpretation of how ETFs stack up against each other. In March, Chicago-based Morningstar started grading ETFs using the same star system that’s helped it to dominate mutual fund research. Lipper offers another set of ETF rankings, as do up-and-coming boutiques like New York City’s AltaVista Independent Research and Buyside Research of Darien, Conn.

Aside from an occasional industry report, in-depth analyst coverage of ETFs was virtually non-existent until recently. While this is changing as dedicated research emerges, many funds are so new they lack the necessary performance history to receive an official grade. Another hurdle has to do with the eccentric nature of ETFs, which resemble mutual funds, but trade like stocks. How should the analysts treat them?

As a result of this confusion, no uniform ETF scoring system has yet emerged and rankings can vary widely. The process of making sense of a fund’s rating starts with understanding the angle from which an analyst views the ETF universe. Firms like Lipper and Morningstar, entrenched in producing mutual fund research, are more inclined to see ETFs from a mutual fund perspective. In contrast, AltaVista and Buyside Research analyze ETFs in a similar fashion to individual stocks, focusing on important financial metrics such as P/E ratios, earnings momentum and valuation multiples.

Looking at the “Big Three”

Among the fund-oriented approaches, Lipper ratings focus on an ETF’s total return, consistency of return, tax efficiency, capital preservation and fund expenses by assigning each aspect of the fund a score between 1 and 5, with 1 representing the highest score and 5 the lowest. Attention to tax efficiency and fund expenses, in particular, is distinctive; while large ETF distributions are uncommon, they are possible and can put a drag on after-tax performance. Naturally, tax efficiency is not a factor if the fund is held in a tax-deferred retirement account.

Morningstar’s ETF ratings are based upon risk-adjusted returns over a three-, five- and 10-year time frame. This is similar to the way the firm rates conventional open-end mutual funds; in fact, Morningstar compares ETFs against a peer group of open-end funds in the same category.

Two important aspects of Morningstar’s approach are that risk-adjusted returns of ETFs are based upon their NAV and not open market returns. “NAVs for ETFs are most comparable to open-end mutual funds that operate in a NAV environment,” explains senior fund analyst Dan Culloton. “This also avoids the problem of ETFs with stale pricing due to low trading volume.”

Also, Morningstar adjusts ETF returns to reflect a trading commission of 0.20 percent on the front and back end. This figure represents the estimated average retail commission paid ($20) on a $10,000 investment.

Standard & Poor’s uses a star system similar to Morningstar’s, but rates ETFs based upon its own set of characteristics. Funds in the top 10 percent to 20 percent of their category get four and five stars, whereas funds with lagging performance are assigned only one or two stars. Rankings do not take trading fees into account. However, S&P looks at how an ETF is projected to perform versus its peers over a 6-month time frame.

Since Morningstar and S&P also license their indices to ETF managers, both firms are careful to avoid any perceptions of conflicting interests. Morningstar does not provide analyst opinions for ETFs tracking its indices; S&P provides only quantitative ratings.

Boutiques Jump at the Chance

Not everyone is sold on rating ETFs based upon their historical performance. Michael Krause, president of AltaVista, feels that “past performance of an ETF should have about the same importance as a price chart for a stock. Maybe it’s interesting from a short-term trading or technical perspective, but not very indicative of future performance.”

Instead, Krause argues the investment merit of ETFs should be based upon forward-looking fundamentals. AltaVista provides a table of various valuation metrics such as price-to-earnings (P/E), price-to-book (P/BV), price-to-cash flow (P/CF) and price-to-growth (PEG). Having all of the data figures alongside each other (updated daily on the company’s website) is especially helpful.

Buyside Research takes a pure equity analyst approach to its consideration of ETFs as requested by its Wall Street contacts. “We separate the best from the worst using fundamentals of the funds’ underlying stocks,” explains President James Squyres. “As you would expect, we have different funds in the top and bottom groups compared to Morningstar.”

The firm distributes its reports via financial portals.

While many firms are taking a shot at rating ETFs, TrimTabs Investment Research stands out because it doesn’t bother. Instead, the company tracks the money flows going into and out of ETFs on a daily, weekly, monthly and quarterly basis in order to “offer a glimpse into the thinking of predominantly institutional traders.”

Good Start; More Is Needed

Although each rating service has its strengths and weaknesses, opinions on expenses, tax efficiency and tracking error were hard to come by. Only Lipper provides a score on fund expenses and tax efficiency, and none of the others report on how well ETFs perform tax-wise versus each other. Grades for tracking error (a statistical measure of how well a fund shadows its index) are also noticeably absent. Obviously, some funds track their underlying benchmarks better than others, but the rating services have yet to report this statistic — a titanic omission in the exclusively index-driven ETF universe!

While ETF ratings provide a point of reference, it’s important for advisors to tread carefully. Like anything new, the quality of ETF research still needs time to evolve and improve. For now, the art of ETF analysis is not unlike ETFs themselves. Both have a lot of growing up to do.

Web-Based Resources

www.altavista-research.com: ETFs are given a positive or negative opinion; focus is on fundamentals of ETF stock indices.

www.buyside-research.com: ETFs are graded on a scale from 1 to 4, with 4 being the best; focus is on fundamentals of ETF stock indices.

www.lipperweb.com: ETF ratings are based on total return, consistency of return, tax efficiency, capital preservation and fund expenses.

www.morningstar.com: ETFs are rated using the same star system as traditional mutual funds.

www.standardandpoors.com: ETFs are graded on a scale of 1 to 5, with 5 being the highest score.

www.trimtabs.com: Tracks daily and weekly ETF flows; doesn’t rate funds.