CHICAGO (HedgeWorld.com)–Backfill reporting bias adds on average 4 percentage points annually to hedge fund returns in the TASS Database,* which in turn translates into as much as US$26.1 billion a year in returns that occurred prior to hedge funds joining the database.

In their paper, “A Reality Check on Hedge Fund Returns,” researchers Nolke Posthuma, vice president of research for ABP Investments, and Pieter-Jelle van der Sluis, senior vice president of research for ABP Investments and assistant professor of finance at Free University of Amsterdam, argue that backfill returns constitute a larger portion of total returns in the TASS Database and add more to overall reported hedge fund performance than previously thought.

TASS officials acknowledge backfill bias exists but say this paper, through unsupported inferences and incorrect assumptions, makes the bias out to be worse than it is and worse than previous research would support.

Backfill returns occur when a hedge fund has been operating for some period of time prior to reporting to a database. For example, a hedge fund may be up and running and generating returns for a year or more before it starts reporting to a database. Then the database agrees to include in its own historical data the past hedge fund returns from when the fund was not part of the database.

The bias comes in because often hedge fund managers with good past returns are the ones who want to be included in a database. Fund managers with poorer past performance often do not want to report to a database and have their returns compared to better-performing peers. Some argue that including funds with good past returns and adding those returns to the database history, while not including poorer-performing funds or funds that failed, skews overall database returns upward.

The ABP researchers examined 3,580 hedge funds that reported results to the TASS Database between January 1996 and December 2002. They calculated annualized returns for 10 styles and one “unknown” category comprising funds whose style could not be determined from TASS information. Their calculations comprised backfilled and non-backfilled returns in three scenarios: one in which funds that stopped reporting were assumed to have no future negative returns, one in which those funds were assumed to drop another 50% in value after they stopped reporting and one in which funds were assumed to drop 100% in value after they stopped reporting.

What they found was that the average length of the backfill periods, or so-called “instant histories,” stretched for roughly 34 months, longer than previous studies had indicated. They also found that more than half of all returns in the TASS Database were backfilled returns, meaning funds’ track records were, on average, longer than their actual histories in the database.

To test backfilled versus non-backfilled returns, the researchers created a fictional investor who only invested in hedge funds during the periods those funds reported results to the TASS Database. What they found was that if funds that stopped reporting to TASS were assumed to have no additional negative returns after they stopped reporting, the investor would have earned a return of 6.4% annualized across all funds over the period from January 1996 to December 2002.

By contrast, annualized return results for all funds–including backfilled returns and again assuming no additional negative returns for funds that stopped reporting–were 10.7% over the same time period. That’s a difference of 4.3 percentage points across all styles.

The researchers said in their report that the 4.3 percentage point difference is their “most conservative estimate of backfill bias.” Other scenarios that assume further negative returns for funds that stop reporting only increase the backfill bias, according to the report.

When the pair assumed that funds that stopped reporting dropped another 50% in value after they stopped reporting and before investors could redeem their assets, backfilled returns fall to 7.34% annualized across all funds between January 1996 and December 2002 while non-backfilled returns drop to 0.11%, a difference of 7.24 percentage points.

When it was assumed that funds that stopped reporting lost 100% of their value after they stopped reporting, backfilled returns fell to 4% while non-backfilled returns fell to -6.2%, a difference of 10.2 percentage points.

Different hedge fund styles showed either more or less backfill bias. Long/short equity strategies generally had the highest backfill bias of any clearly identified strategy in each of the above three scenarios. Event-driven strategies generally had one of the lowest backfill biases, according to the report.

On a potentially positive note, the number of backfilled funds as a percentage of all reporting funds fell through the years from 69% in 1996 to 13% in 2002, according to the report.

Stephen Jupp, vice president and director of quantitative research for Tremont Capital Management Ltd., Rye, N.Y., which owns and operates the TASS Database, said he found various problems with the report written by the researchers. Chief among them was the assertion that all backfill returns could be discounted.

By ignoring all returns generated prior to joining the TASS Database, the researchers are in some cases throwing out several years’ worth of returns generated a year or more after the fund’s inception, Mr. Jupp said. Previous research has shown that generally the first 12 months of a new fund’s returns can be discounted, since those early returns tend to be better than returns going forward. But discounting an average of 34 months’ worth of returns is too much, Mr. Jupp said, and in a number of cases needlessly ignores good performing funds simply because they were not part of the TASS Database when they generated the good returns.

Additionally, Mr. Jupp said there was no basis in reality for any assumption that a fund that stops reporting to the TASS Database would lose 50% or 100% of its value after it stops reporting, making those return comparisons are invalid.

Finally, Mr. Jupp took issue with the researchers’ use of equal weighted returns, saying it would have been more accurate to asset weight them.

The abstract and full study can be found at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=438840.

*TASS Research is the information and research unit of Tremont Capital Management Inc., Rye, N.Y., which is a minority investor in and strategic partner of HedgeWorld