From the September 2002 issue of Investment Advisor • Subscribe!

Prospecting StarMine

Wondering about the accuracy of earnings reports?

These days, finding a good analyst is like trying to find a needle in a haystack. Or like being suddenly cast adrift from your trusted, newfangled electronic equipment, only to have to navigate your way by the stars to a safe and trusted harbor--it's an easy way to miss the entrance and end up in a quagmire.

StarMine leads the way out of this predicament, which has all the signs of short-circuiting investor confidence for many moons to come.

A quagmire, in addition to its definition as a hazardous situation, is also a soft boggy or marshy area that gives way underfoot. I cannot think of a more apt description of the world of earnings estimates. Fiscal quarterly and yearly consensus earning estimates (the average of a pool of individual earnings estimates) are a driving force of the market, and the numbers, while easy to find (e.g., finance.yahoo.com), are not as much bogey as they are boggy.

Even the most sophisticated analyst or manager can get lost in the bog of individual estimates. How so? Earnings estimates are submitted by sell-side analysts at any time during the quarter. They may be revised and updated based on guidance from the company, or not. They can be calculated by seasoned analysts or fresh-faced kids straight from B-school with no real familiarity with the industry. They can be affected by a close relationship with the CFO, or a vacation or maternity leave, or the stripers running off the coast of Cape Cod on a great day for fishing. And naturally, they can also be affected by investment banking and other business relationships.

Thompson Financial, the company that supplies both First Call and I/B/E/S earnings estimates, is excellent at collecting all the data, but the delivery system leaves a lot to be desired. Most analysts and portfolio managers receive their estimates through third-party vendors who can package and spin the numbers for easier decoding. Premium vendor Factset Research Group leads the industry with sophisticated error-checking, screening, and reporting tools, but until recently, customization was a laborious process generally used by only the most fastidious of analysts.

In 1998, Joseph Gatto recognized the need to make sense of the system and applied his extensive experience in strategic management consulting to this quagmire; he not only found the bottom, but transformed the boggy mess into a clear, concise system that ranks analysts based on past performances and then recalculates consensus estimates based on an analyst performance-weighted system. StarMine, Gatto's privately held San Francisco-based company, now provides this information to over 100 buy-side institutional firms and a number of sell-side firms, most notably Merrill Lynch, which uses StarMine's ratings to review its own analysts' performances. In May of 2001, StarMine partnered with Bloomberg News Services to deliver earnings alerts through Bloomberg's desktop network, and in December 2001, StarMine was added to FactSet's list of databases, coupling StarMine's analytical tools with FactSet's screening and formula library--the backbone of FactSet's corporate identity.

StarMine uses a proprietary metric system (relative accuracy score, or RAS) that compares analysts to their peers based on profitability of their stock recommendations and accuracy of their earnings estimates. Among the considerations are absolute forecast errors, errors compared to other analysts, variance and timing of estimates, and most important, actual earnings of the stock. StarMine then assigns each analyst his own RAS score, with different ratings for stockpicking and earnings, while recognizing that a given analyst might be more accurate on a few core stocks he or she follows most closely. Notably, if an analyst's estimates are both different and more accurate than other analysts' estimates, they receive a high score; herd followers receive lower scores, and the perennially wrong score the lowest. This data is computed and revised daily as ratings changes and actual earnings figures come in; thus an analyst's call on one stock can affect the estimate for the other stocks he or she covers. (While StarMine Professional is a premium service, the www.starmine.com site shows top analysts, "surprise" earnings predictions, and SmartEstimates for free to registered StarMine Investor users.)

StarMine then combines this analyst rating system with other proprietary formulas (more recent revisions of the best analysts are particularly heavily factored) to calculate a "SmartEstimate" for individual stocks' earnings. According to SmartMine, these SmartEstimates outperform consensus earning estimates in predicting companies' future earnings; when SmartEstimates differ from the consensus, their success rate at predicting the direction of surprises is about 75%. This shouldn't come as a surprise, since you are no longer relying on all the analysts who follow the stock to come up with an average earnings estimate; you are only relying on analysts with a proven track record--or rather, you are relying on each analyst to exactly the extent to which his or her record has been proven. Suddenly, the quagmire is gone.

In addition to portfolio managers, StarMine's target audience is quantitative groups who trade on momentum. They will be both the toughest critics of what can be rather sloppy First Call and I/B/E/S data (Thompson relies on third-party vendors' data integrity teams to verify the numbers, and StarMine might be wise to investigate using FactSet's famously well-scrubbed numbers to generate their rankings if data integrity becomes a problem). In the wake of the market's fallout and growing distrust in investment bankers and their analysts, StarMine is able to quickly and easily identify those analysts who do their job well and those who have a history of running aground.

Reprints Discuss this story
This is where the comments go.