Swedroe: How Survivorship Biases Happen

Not considering closed funds gives investors a flawed picture of performance.

One of the problems with some of the research on active management, and also mutual fund performance rankings, such as Morningstar’s popular percentile rankings, is that they contain survivorship bias—they consider only funds that have survived the full period. This contrasts with data provided through the S&P Dow Jones Indices Versus Active (SPIVA) scorecards, which is free of survivorship bias. Its rankings consider all funds that began the period, whether they survived or not.

Because we know mutual funds firms don’t merge or shut down successful funds, we can conclude that funds they close almost certainly had poor performance. The problem for investors selecting mutual funds today is that they are choosing from a list that excludes losing funds that have been either shuttered or merged out of existence to make their poor performance disappear. Understanding how survivorship bias impacts the odds of success in selecting actively managed funds that will outperform in the future is an important issue.

A Vanguard Study
Vanguard’s research department took a look this problem in a study that covered the period 1997 through 2011. I believe many investors will be surprised at how big a problem survivorship bias actually is. Following is a summary of its findings:

• Just 54% of funds managed even to survive the full 15 years. The rest (2,364 funds) were either liquidated or merged into another fund in the same fund family—in some cases more than once.
• As you would expect, the leading cause for a fund to fail was underperformance. Funds that failed experienced negative cash flows at the time of closure, as investors responded to the poor performance.
• Investors had a 79% chance of picking a fund that underperformed, was liquidated or had a life cycle too convoluted for researchers to disentangle. For large growth funds, the odds of failure were even higher—82%. For large value funds, they were slightly better—73%.

Vanguard also looked at what happened to stock funds that were merged, rather than closed. After the mergers, 73% of funds underperformed. Accounting for both pre- and post-merger periods, 87% of funds underperformed for the full 15 years. In seven of the fund categories, at least 99% of these merged funds underperformed. For bond funds, the data was even worse. For government bond funds, the failure rate was 89%; for corporate bond funds, it was 99%; and for high-yield funds, it was 100%. In other words, while merging a poorly performing fund might provide hope for the investor, it doesn’t improve the odds of outperforming in the future.

Vanguard’s findings confirm prior research. For example, a study by Lipper found the following: In 1986, the then-existing 568 stock funds returned 13.4%. By 1996, that 1986 performance had “magically” improved to 14.7%. This 1.3 percentage point improvement was the result of the disappearance of 24% of the original funds, and the fact that only the 1986 performance of funds still in existence 10 years later was used in the new computation.

Active Vs. Passive

We’ll now examine how survivorship bias impacts the performance rankings of active funds compared to funds from two leading providers of passively managed funds: Dimensional Fund Advisors (DFA) and Vanguard. The following table covers the 15-year period from 2002 through 2016. (Full disclosure: My firm, Buckingham, recommends DFA funds in constructing client portfolios.)

The third column shows Morningstar’s percentile ranking, which has a survivorship bias problem, as it only includes funds that were in existence for the full period. Note that a ranking of 1 is the best performance. The fourth column shows the number of funds in existence at the start of the period, as well as how many of them survived the full period, and the ratio.

We then make the assumption (which Vanguard’s study demonstrates is likely very close to the reality) that all the mutual funds that didn’t survive the period performed poorly, underperforming simple index strategies. The final column adjusts the ranking to take into account the survivorship bias. Thus, if there were 100 funds at the start of the period but just 50 at its end, a fund with a Morningstar ranking of 40 would improve to 20 (50% x 40) after considering the survivorship bias problem.

Using Morningstar’s unadjusted rankings, we see that Vanguard’s funds had an average ranking of 36 and DFA’s funds had an average ranking of 16. Once we account for survivorship bias, Vanguard’s average ranking improves from 36 to 21 (a 42% relative improvement), meaning that it outperformed 79% of actively managed funds. DFA’s average ranking improved from 16 to 10 (a 38% relative improvement), meaning it outperformed 90% of actively managed funds.

Keep in mind that even these figures understate the relative performance for taxable investors. The higher turnover of actively managed funds tends to make them less tax efficient. On an after-tax basis, the odds of success would be much worse. For example, the study “How Well Have Taxable Investors Been Served in the 1980s and 1990s?” found that the average fund underperformed its benchmark by 1.75% per annum before taxes, but by 2.58% on an after-tax basis.

Furthermore, just 22% of funds beat their benchmark on a pretax basis. The average outperformance was 1.4% and average underperformance was 2.6%. However, on an after-tax basis, just 14% of the funds outperformed. Average after-tax outperformance was 1.3%, while average after-tax underperformance was 3.2%. Thus, the risk-adjusted odds against outperformance were about 17:1.

The results make clear that active management is a strategy we could consider “fraught with opportunity.” Year after year, active managers come up with excuses to explain why they failed this time around, and then state that next year will be different. Of course, it never is. In addition, evidence is piling up that the odds of winning the game of active management have been persistently declining.

Less Alpha To Go Around

In our book, “The Incredible Shrinking Alpha,” my co-author, Andrew Berkin, director of research at Bridgeway Capital Management, and I present evidence demonstrating that while 20 years ago about 20% of actively managed funds were generating statistically significant alpha, today that figure is about 2%. And that’s before taxes.

We also present the evidence and logic behind four themes that explain what we believe is an inexorable trend: academic research has been converting what was once alpha into beta; the pool of victims that can be exploited has been shrinking; competition has been getting tougher; and the supply of dollars chasing alpha has increased (though this may have begun to finally change).

The bottom line is that, while it is possible to choose actively managed funds that will outperform, the odds of doing so are so poor that it isn’t prudent to try. Making the matter even worse is that investors typically don’t build portfolios that consist of a single fund. Diversifying their portfolios requires owning multiple funds. That pushes the odds much lower of successfully building a portfolio of actively managed funds that will outperform (I address this issue in a 2011 article).

The good news is that investors are waking up to the reality. In October 2016, The Wall Street Journal reported that, according to Morningstar data, “although 66% of mutual-fund and exchange-traded-fund assets are still actively invested … those numbers are down from 84% 10 years ago and are shrinking fast.”

Larry Swedroe is the director of research for The BAM Alliance, a community of more than 140 independent registered investment advisors throughout the country.