A study released Monday by Vanguard shows that many new indexes used to build ETFs generally do well when they are “back tested,” or applied to market data from before their inception. However, only about half beat their benchmarks after the ETF is introduced to investors, the research found.
“When historical index performance is hypothetical, the notion that past performance may not indicate future results is especially true,” said Joel Dickson, one of the study’s authors and a principal in Vanguard’s Investment Strategy Group, in a press release.
“Are back-filled data a reliable indicator of how the index will perform after it is launched? Our answer is a decided no,” the report stated.
The index average for the indexes studied by Vanguard outperformed the broad stock market at an annualized rate of 10.31%, as measured by five years of back-filled data. Yet, they underperformed the market at an annualized rate of –0.93% over the five years following the index live date.
The rapid growth in ETFs has gone hand in hand with a proliferation of target benchmarks, according to the study, entitled “Joined at the Hip: ETF and Index Development,” which notes that about $1.2 trillion was invested in some 1,400 U.S.-listed ETFs as of March 31, according to Strategic Insight’s Simfund. Today, U.S.-listed ETFs seek to track more than 1,000 different indexes.
Vanguard researchers say that more than half of these indexes had been in existence for less than six months before the launch of an ETF that seeks to track it. They examined data for 370 indexes that had at least six months of back-filled data and six months of live data from 2000 through 2011 in their study.
This research revealed that while 87% of the indexes outperformed the broad U.S. stock market for the time in which back-tested data were used, only 51% did so after the index was launched.
Securities regulations prohibit the use of back-tested performance for most U.S. mutual fund and ETF providers. But index providers are not held to the same rules, according to Vanguard, and this can make it difficult for investors to discern which data are hypothetical and which are live.
Vanguard researchers found that 20 different weighting methods and 27 different selection criteria exist for indexes that seek to be tracked by ETFs. Furthermore, the percent of ETF products based on alternative-weighting approaches jumped from 5% in 2000 to almost 50% in 2011, according to IndexUniverse.
These alternative-weighting approaches have not consistently produced excess returns after accounting for size and style exposures, the researchers say, pointing to industry research.
The fund giant adds that it is the third-largest U.S. ETF provider by assets under management and had cash flows of $28.8 billion into its U.S.-based ETFs through June 30. The firm manages 64 ETFs with a total of $209 billion in ETF assets.