FINRA’s Wrongheaded Ruling On Backtesting

May 14, 2013

A FINRA ruling on backtesting for new ETFs serves as a reminder of how not to invest.

In late April, FINRA made an interesting ruling regarding the marketing of backtested index data in the launch of new ETFs. I’m not so convinced that investors are better off.

Prior to the guidance letter FINRA posted in April, the use of backtested index data in marketing materials and communications of newly launched ETFs was prohibited. Executives in charge of marketing new ETFs that tracked lesser-known indexes were left with pointing potential investors to index-provider websites.

The strategy of directing potential investors to backtested index data made a huge difference. Last year Vanguard did some research on the matter, comparing inflows of ETFs tracking a backtested index, and those tracking an index without backtested data. Vanguard found that in all asset classes except fixed income, the ETFs tracking backtested indexes received greater inflows.

It’s no surprise; it’s much easier to market something that has a semblance of history behind it.

However, the latest FINRA ruling allows ETF issuers to use backtested index data in ETF marketing materials to institutional investors. Yes, you read that correctly. FINRA is still prohibiting the marketing of ETFs in such a fashion to retail investors.

FINRA made the decision specifically as a response to an ETF distributor’s request for interpretive guidance regarding the use of such data because it had institutional clients that wanted to see the backtested data. But does this really make any sense?

I can understand institutional clients wanting to get a better gauge of an ETF’s strategy by looking at historical index data. That said, it’s also clear to me that institutional investors can make the same mistakes that retail investors make in interpreting past performance as a promise of future returns.

On top of that, backtested index data can be rife with flaws, as they often apply today’s methodology to yesterday’s market. As Brendan Conway points out, one can easily imagine the creation of a backtested financial sector index that omits the likes of Lehman Bros. and Countrywide Financial.

Pandora's Box

Frankly, I’m a little skeptical of the idea that only institutional clients will see these marketing materials. And really, if we’re discussing the fairness of markets, if any information on these newly launched ETFs is going to be shown, then all investors should see them. There’s no point in trying to separate investors into groups—Pandora’s Box is already open.

What I’m most concerned about is what this means in the marketing of existing and new ETFs—especially when you consider the growth in popularity of alternative index strategies that stray from the plain-vanilla market-cap-weighted indexes of yesterday.

Even if backtested index data look amazing, they do absolutely nothing in terms of predicting performance of a newly launched ETF. A number of things can eat into the expected performance— whether it’s the expense ratio or the optimization strategy that the ETF’s portfolio manager may choose to implement. That’s especially true in the case of newly launched ETFs. There’s no telling how the fund will actually fare when compared with its underlying index.

Interestingly, in Vanguard’s research paper, the fund firm found that backtested indexes outperformed the market in the five years prior to the ETF’s launch, but underperformed in the next five years.

Backtested data may offer some insight, but it’s something to be taken with more than a few grains of salt.

Understanding the ETF’s underlying costs, optimization strategy, and general investment thesis are much more valuable than simply looking at a chart.

Investors need to take charge of their own due diligence—whether institutional or retail. Just remember, past performance is truly no guarantee of future results.



Find your next ETF

Reset All