On Magical Thinking And Investing*

February 28, 2012

On Magical Thinking And Investing

Readers of my books often ask: “Given that the evidence against active management and for indexing (or passive investing) is so overwhelming, why do the majority of investors keep playing a loser’s game?” I offer some explanations for this phenomenon.

First, the education system has failed the public. Unless an investor obtains an MBA in finance, it’s likely he or she hasn’t taken a single course in capital markets theory. The result is that investors get their “knowledge” about investing from the very people—Wall Street and the financial media—who don’t have their interests at heart because the winning strategy for them is for investors to play the game of active investing. Second, despite the importance of the issue, the public seems unwilling to invest the time and effort to overcome the failings of the education system. Instead of reading books like mine or John Bogle’s or William Bernstein’s, they would rather watch CNBC—to hear some guru’s forecast—or some reality TV show.

The third explanation is what we might call the Lake Wobegon effect: the need and/or desire to be above average. This seems especially true of high-net-worth people, especially when it comes to investments. Wall Street preys on that need. You hear the repeated lie that goes something like this: “Indexing is a good strategy, but it gets you average returns. You don’t want to be average. We can help you do better than that.” The truth is that indexing gets you market returns. And because it does so with lower costs and greater tax efficiency, by definition, you earn above-average returns—as long as you have the discipline to stay the course. This is about the only guarantee there is in investing.

Ignoring The Evidence
Why do people ignore all the evidence? Why do so many investors continue to play a game that while it’s possible to win, the odds are so poor it’s simply not prudent to try? Kathryn Schulz provides us with some fascinating insights that help explain this phenomenon, one that allows Wall Street to transfer tens of billions every year from the pockets of investors to its own pocket.

Schulz is the author of the book “Being Wrong.” She explains that most of us go through life assuming that “we are basically right, basically all the time, about basically everything.” And that “our indiscriminate enjoyment of being right is matched by an almost equally indiscriminate feeling that we are right. Occasionally, this feeling spills into the foreground, as we … make predictions or place bets [or make investments].” She goes on to explain that often this confidence is justified as we “navigate day-to-day life fairly well …” This suggests that we’re right about most things. Schulz’s book, however, is about being wrong and what happens when our convictions collapse around us—we often feel foolish and ashamed as error is often associated with “ignorance … psychopathology, and moral degeneracy.”

Schulz noted that we tend to view errors as things that happen to others, yet somehow we feel that it is implausible that they’ll happen to us. She believes this is because “our beliefs are inextricable from our identities,” and “we’re so emotionally invested in our beliefs that we are unable or unwilling to recognize them as anything but the inviolate truth.” She notes that “ . . . we have a habit of falling in love with our beliefs once we have formed them. [Such as ‘active management is the winner’s game’].” And that explains why “being wrong can so easily wound our sense of self.” It explains why we experience cognitive dissonance—the uncomfortable feeling and/or anxiety we feel when someone disproves a long-held belief. It also explains why we ignore evidence, even when it is compelling, and why we resist change.


Margaret Heffernan, author of “Willful Blindness,” provided some additional insights into the behavioral problems surrounding the issue of being wrong. Willful blindness is a legal concept—you can be held responsible even if you weren’t aware, but could and should have known something was wrong and decided not to see it. The claim of not knowing is not a sufficient defense. Heffernan notes: “The law doesn’t care why you remain ignorant, only that you do.” Corporate executives greedy for compensation, politicians who vote for legislation knowing it will never work, and auditors who turn blind eyes to findings because they don’t want to lose their client’s business all make destructive blunders because of willful blindness.

Heffernan observes: “We mostly admit the information that makes us feel great about ourselves, while conveniently filtering whatever unsettles our fragile egos and most vital beliefs.” Her views on the subject are very similar to those of Kathryn Schulz. Heffernan states that while “love is blind, what’s less obvious is just how much evidence it can ignore.” The conclusion she reaches is that fear of change and conflict can blind us to evidence—and so can the power of the almighty dollar.

Schulz believes that perhaps the greatest error of all is the tendency to think that others—not us—make errors because “wrongness” is a vital part of how we learn and change. When discussing the academic evidence on passive investing as the winning strategy, I have frequently observed that no one likes to be told they’ve been doing something wrong all their lives. To address this problem, I point out that I used to be an active investor. However, I did what all smart people do when they learn they are mistaken in their beliefs—they don’t repeat the mistake because that is the behavior of fools.

Another important insight provided by Schulz is that our ability to forget our mistakes is keener than our ability to remember them. During her research, Schulz met many people who said she should interview them as they make mistakes all the time. Yet, when asked to give specific examples of their mistakes, they were hard-pressed to come up with any. If you want to have some fun, try asking your friends to give example of their mistakes. The inability to remember mistakes leads to overconfidence, which in turn leads to other mistakes, especially investment mistakes—such as taking too much risk and failing to diversify—which can be very expensive.

Schulz notes that we think our beliefs are based on facts and reason, and that we are rational. When we reject the beliefs of others, we think we possess good judgment. She calls this the Ignorance Assumption. She notes that at times it holds, but not always. And when the Ignorance Assumption fails, when people stubbornly refuse to agree with us even after we enlighten them, we move on to the Idiocy Assumption—they know the facts but they just don’t have the intelligence to comprehend them. And when that fails, there is always the Evil Assumption—people know the truth but have turned their backs on it.  

Schulz concludes that “mistakes disturb us in part because they call into question not just our confidence in a single belief, but our confidence in the entire act of believing. When we come to see one of our own past beliefs as false, we also glimpse, for a moment, the persistent structural possibility of error: our minds, the world, the gap between them—the whole unsettling shebang.” This is why established institutions such as the Catholic Church fight so hard to suppress ideas that are contrary to doctrine, such as the Earth is the Center of the Universe. People were burnt at the stake for disagreeing. And the famous astronomer Galileo spent the last years of his life under house arrest for stating what the Church’s own scientists already knew to be true. They fight so hard because they know if people learn they are wrong about one thing, perhaps they might get the idea that they are wrong about the whole shebang. This is why Wall Street and much of the financial media fights so hard to suppress and reject the evidence that passive investing is the winning strategy—for them it is economic suicide.

Returning to the beginning of this talk, there is yet a fourth explanation for why more investors aren’t passive. The authors of the wonderful book “Mistakes Were Made (But Not By Me),” social psychologists Carol Tavris and Elliot Aronson provide us with this reason: “Most people, when directly confronted with proof that they are wrong, do not change their point of view or course of action but justify it even more tenaciously. Politicians, of course, offer the most visible, and often tragic, examples of this practice . . . We stay in an unhappy relationship or merely one that is going nowhere because, after all, we invested so much time in making it work.

Tavris and Aronson explain: “Self-justification has costs and benefits. By itself, it’s not necessarily a bad thing. It lets us sleep at night. Without it, we would prolong the awful pangs of embarrassment. We would torture ourselves with regret over the road not taken or over how badly we navigated the road we did take. We would agonize in the aftermath of almost every decision . . . Yet mindless self-justification, like quicksand, can draw us deeper into disaster. It blocks our ability to even see our errors, let alone correct them. It distorts reality, keeping us from getting all the information we need and assessing issues clearly.”

Investors, relying on the past performance of active managers, and rankings like Morningstar’s ratings, hire managers, then eventually fire most of them and repeat the process. They do so without ever asking: “What am I doing differently in the selection process so I don’t repeat the mistake I made last time?” In a triumph of self-justification, they end up doing what Einstein said was the definition of insanity—doing the same thing over again and expecting a different outcome.

Why Do The Experts Get It Wrong?
I would like to move on now to discussing why experts keep failing us, why they keep getting it wrong. As you know, many investors have seen their portfolios devastated despite having followed the advice of experts. They are left to wonder, “What went wrong?” The answer is that the strategy of following the advice of “future tellers” disguised as experts is the wrong strategy. As investment legend Warren Buffett put it: “A prediction about the direction of the stock market tells you nothing about where stocks are headed, but a whole lot about the person doing the predicting.”

David Freedman, in his book, “Wrong,” shows that experts are often the reason we get into big messes, whether the field is medicine, investing, science, psychology, raising children, dieting or business management. With often frightening examples, he exposes the biases and career pressures that often dangerously influence the ways in which experts arrive at their advice. He explains: “Most of us think of scientists as being devoted to uncovering truths, not pumping their career prospects. Less formal experts . . . don’t enjoy that sort of halo.”

Freedman notes that a wide range of economists and even mathematicians, as well as many nonscientist financial experts, have been demonstrating quite clearly for about a century that no matter what technique you use to pick stocks, you’re not likely to beat the market; that “many of us still put our faith in, not to mention bet our life savings based on, the advice of, say, a screaming, bouncing, bell-ringing television personality who claims to have special insight into the movements of stocks, is, I think, a sharp illustration of how some experts can ride straight-out irrationality to great personal success.”

Freedman goes on to add: “The simple fact is that most informal experts can spew out conclusions without much fear of being intercepted by wiser or more careful parties. Who’s filtering the recommendations of investment gurus? . . . in the short run, most informal experts can get away with quite a bit, and do all the time.”

Freedman provides a great example of how people react to advice. He presents the case of an individual with back pain, who visits two doctors. Each reviews the patient’s MRI. The first doctor states that he’s seen many similar cases and that it’s hard to say exactly what is wrong. It’s hard to predict what will work for any one person. He suggests they try Treatment A first, and then go on from there. The second doctor states that he knows exactly what is wrong and what to do. Which doctor to choose? Almost always people will choose the latter. Yet, that might very well be the wrong choice. While we want certainty, it rarely exists, and it certainly doesn’t exist in the investment world, where so much of returns are explained by unforecastable events such as Mideast revolutions, Japanese earthquakes and tsunamis, the attack on the World Trade Center buildings, etc.

Foxes And Hedgehogs
Most of us want certainty, even when we know, logically, that it doesn’t exist. With investing, it is a desire to believe there’s someone who can protect us from bear markets and the devastating losses that can result. Yet, political scientist Philip Tetlock demonstrated in his outstanding book, “Expert Political Judgment: How Good Is It? How Can We Know?”, that even professional economic forecasters don’t make accurate forecasts with any persistence.

Tetlock found the so-called experts who make prediction their business—who appear as experts on television and talk radio, are quoted in the press, and advise governments and businesses—are no better than the proverbial chimps throwing darts. He divided forecasters into two general categories: foxes who draw on a wide variety of experiences and for whom the world cannot be boiled down to a single idea; and hedgehogs who view the world through the lens of a single defining idea. The following are some of Tetlock’s most interesting findings:

  • What distinguishes the worst forecasters from the not-so-bad is that while hedgehogs are more confident, they are wrong more often than foxes. Unfortunately, overconfidence is an all-too-human trait.
  • Optimists tend to be more accurate than pessimists. Keep this in mind the next time you read a doomsday forecast.
  • What experts think matters far less than how they think. We are better off turning to the foxes who know many little things and accept ambiguity and contradiction as inevitable features of life rather than turning to hedgehogs who reach for formulaic solutions to ill-defined problems.
  • Experience, profession and privileged information make virtually no difference to a forecaster’s accuracy. The only predictor of accuracy was fame, which was negatively correlated with accuracy: The most famous—those more likely feted by the media—made the worst forecasts.
  • In general, subject matter expertise translates less into forecasting accuracy than it does into overconfidence—and the ability to spin elaborate tapestries of reasons for expecting “favorite” outcomes.
  • Like ordinary mortals, experts fall prey to the hindsight effect—they claim they know more about what was going to happen than they actually knew before the fact. This systematic misremembering of past positions may look strategic, but the evidence indicates that people sometimes truly convince themselves that they “knew it all along.” Hindsight bias causes overconfidence.
  • The “market place of ideas” can fail because consumers may be less interested in the dispassionate pursuit of truth than in buttressing their prejudices.


One of my favorite sayings is that there are three types of investment forecasters: those who don’t know where the market is going; those who know they don’t know; and those who know they don’t know but get paid a lot of money to pretend they do. In other words, they are playing an entirely different game.

The lesson we should take from the research is that we should be wary of so-called expert advice. David Freedman cautions to be wary of advice that has the following characteristics:

  • Is clear-cut, free of doubt and actionable;
  • Is universal, a one-size-fits-all solution;
  • Upbeat and palatable; and
  • Filled with dramatic claims and stories, and specifically numbers that add an illusion of precision.

A Way Forward
So this is a call for action. If you have been using active strategies and have failed to outperform appropriate risk-adjusted benchmarks, or the fund managers you have hired have failed to do so, or if your advisor using active strategies has failed to do so, ask yourself:

  • What will I be doing differently in the future to ensure a different outcome? If I cannot identify anything, then why do I think the outcome will be different?
  • Why do I think I will succeed where others (institutional investors) with far greater resources and other advantages (such as lower costs and/or no taxes to pay) have failed with such great persistence? What advantages do I have that will allow me to succeed?


Finally, it’s my experience that the vast majority of investors don’t even know what their returns have been relative to appropriate benchmarks. One reason is that Wall Street doesn’t want you to know—if you did, you might stop making it rich. Another might be that the truth would be too painful, so investors themselves don’t want to know. But you should know. Without such information, there is no way to know if your strategy is working.

If you don’t know, my suggestion is to take your portfolio to a fiduciary advisor, one that uses passive strategies, and ask them to analyze your portfolio and show you how it has performed relative to appropriate benchmarks. It might just turn out to be the best “investment” you have ever made.


Larry Swedroe is the director of research for the BAM Network of financial advisors and a well-known advocate of index-based investment strategies. The foregoing is an adapted version of his remarks to registered investment advisors in a conference call.

*This article is solely available online; it is not included in the print version.


Find your next ETF

Reset All