Beating the market was always hard, but who knew it’s literally becoming harder?

Editor’s Note: This article is an excerpt from The Incredible Shrinking Alpha, an upcoming investment book co-authored by Larry Swedroe and Andrew Berkin. Its expected publication is in early 2015. Also, Swedroe will deliver a keynote address by the same name at the Inside ETFs conference Hollywood, Florida in January 2015.

Baseball’s “modern era” began in 1903. Prior to that time, foul balls that weren’t caught weren’t considered strikes, giving hitters a major advantage. From 1903 through 1941, seven different players achieved a batting average of over .400 a total of 12 times—Ty Cobb and Rogers Hornsby each did it three times, and George Sisler did it twice.

The last National League player to hit .400 was Bill Terry of the New York Giants, who hit .401 in 1930. The last American League player to do so was Ted Williams, who was known as the Splendid Splinter, of the Boston Red Sox. He hit .406 in 1941. So it’s been more than 70 years since anyone has hit .400.

What Gives?

Today’s baseball players are superior athletes. They are bigger, faster, stronger, use improved training techniques and have better diets. That presents us with an apparent anomaly. Why have superior athletes been unable to achieve once in the last 73 years what was accomplished 12 times in just 39 years?

Part of the explanation is that on defense, today’s players are also superior and have better equipment. On the other hand, the strike zone is much smaller today, giving hitters a major advantage. Statistical analysis, however, provides us with the answer.

It has to do with what is known as the “paradox of skill.”

The Paradox of Skill

In many forms of competition, such as chess, poker or investing, it’s the relative level of skill that plays the more important role in determining outcomes, not the absolute level. The “paradox of skill” means that even as skill level rises, luck can become more important in determining outcomes if the level of competition is also rising.

The result is that, as the average skill increases, it becomes more difficult to outperform by large margins. In other words, the standard deviation of outcomes narrows.

And, by definition, there’s no tendency for luck (either good or bad) to persist. In baseball, that means players are likely to have “hot” and “cold” months purely as result of lucky (or unlucky) outcomes, but that isn’t likely to be the case over a full year. That’s why you don’t see .400 hitters any more. Let’s look at the statistical evidence.

A Few More Baseball Stats

In the first two decades of the modern era, the average hitter batted about .250-.260. During that period, the .400 mark was reached four times. And since 1950, batting averages have been fairly stable in that same .250-.260 range. Yet no one has hit .400.

Why?

In 1921, the standard deviation of batting averages was 40.6 points. By 1941, the year of the last .400 hitter, the standard deviation had fallen to 32.5 points. By 2003, it had fallen to just 26.1 points. If we assume the average player hits .260, then by 2003, hitting .400 had become a more than five standard deviation event.

That means the chances of a player hitting .400 in a given year are below 0.1 percent, or less than 1 in 1,000. In other words, no one hits .400 anymore because the average baseball player of today is far better than the average baseball player of a hundred years ago.

In Basketball, Too

We can see similar evidence when looking at basketball. For example, beginning with 1959-1960, Wilt “The Stilt” Chamberlain had three seasons when he averaged more than 25 rebounds a game. He also had 11 seasons in a row when he averaged more than 22.

Beginning with 1957-1958, Bill Russell had 10 seasons in a row when he averaged more than 20 rebounds a game. And beginning with 1958-1959, he had seven seasons in a row when he average more than 23.

While there’s no doubt that today’s professional basketball players are superior athletes to those who competed 50 years ago, the leading rebounder in the NBA last year was DeAndre Jordan, who averaged just 13.6 rebounds per game. And just four players last year averaged at least 12.

From Chamberlain To Jordan

Chamberlain also averaged more than 50 points per game in the 1960-1961 season. Since that time, the highest average by a player not named Chamberlain was the 37.1 points per game. That feat was accomplished by Michael Jordan in the 1986-1987 season. And last year, Kevin Durant led the league with an average of 32 points per game.

In Asset Management, Too

The paradox of skill means that while today’s athletes are indeed more skillful, they also aren’t as likely to produce the kind of “outlier” returns that the legends of yesteryear achieved.

So it is that we see the same phenomenon in the game of active management. The quest for alpha has become ever-more frustrating because the level of skill of the competitors has been rising.

When it comes to investing, we can think of Peter Lynch and Warren Buffett as the financial equivalents of Wilt Chamberlain and Bill Russell. From 1977 through 1990, Lynch’s Fidelity Magellan fund earned an annual average return of 29 percent. That’s more than twice the 14.0 percent annual average return of the S&P 500 Index.

Similarly, Warren Buffett outperformed the market by more than 10 percentage points a year for about 40 years. Today, active managers are thrilled if they’re able to boast of alpha of even a few percentage points.

Charles Ellis, one of the most respected people in the investment industry, noted the following in a recent issue of the Financial Analysts Journal: “Over the past 50 years, increasing numbers of highly talented young investment professionals have entered the competition. … They have more-advanced training than their predecessors, better analytical tools, and faster access to more information.”

And, according to Ellis, the “unsurprising result” is that “the increasing efficiency of modern stock markets makes it harder to match them and much harder to beat them, particularly after covering costs and fees.”

The Picture Tells The Tale

The above graph shows the rolling five-year average standard deviation of excess returns in U.S. large-cap mutual funds over the last five decades. You should expect to see wide dispersions when there are large differences in the level of skill. But here, the demonstrated trend of declining dispersion in excess returns fits with Ellis’ view that the competition is getting tougher. This is the same phenomenon we saw with baseball batting averages.

More Evidence

Mike Sebastian and Sudhakar Attaluri—authors of the study “Conviction in Equity Investing,” which appears in the Summer 2014 issue of The Journal of Portfolio Management—provide further evidence of the increasing difficulty of succeeding in the quest for alpha.

They found that 20 years ago, about 20 percent of actively managed funds delivered statistically significant alpha. By 2011, the figure was down to less than 2 percent. This finding is consistent with those reported by Eugene Fama and Kenneth French in their 2009 study, “Luck Versus Skill in the Cross Section of Mutual Fund Returns.” They found that there was evidence of statistically significant skill in only the 98th and 99th percentiles.

More of A ‘Loser’s Game’ Than Ever

What, if any, conclusions can we draw from the evidence?

The bottom line is that, just as we have witnessed the disappearance of .400 hitters, we may also have seen the disappearance of superstar investors who were once able to persistently outperform the market by large margins.

Charles Ellis wrote his book, “Winning the Loser’s Game,” in 1998. He presented evidence demonstrating that, while it’s not impossible to win the game of active management, the odds of doing so are so low that it’s not prudent to even try. Hence, the surest way to win the game of active management is to not play.

It’s now 16 years after Ellis’ book was published, and the competition has become even tougher. Thus, we can conclude that it’s getting persistently harder to win what was already a loser’s game.

Larry Swedroe is the director of research for the BAM Alliance, a community of more than 140 independent registered investment advisors throughout the country.