Controlling Regret For

January 01, 2000

The author finds numerous factors such as projected earnings or book-to-price go in and out of favor in the stock market, but nevertheless tend to have persistent and predictable effects on stocks in the S&P 500 for extended periods. He reports good success applying this phenomenon for outperformance in an enhanced index portfolio.

By some estimates, over 20% of institutional assets are now under index management. Reacting to this trend, investment managers have been offering strategies designed to track the passive index, but produce a moderate yet systematic level of outperformance.

 We argue that enhanced indexing as a strategy has merit from a conceptual standpoint. Our experience with enhanced index strategies also demonstrates that it is possible to systematically outperform the passive index, with a level of risk that is virtually identical to that of the index. Moreover, we believe that such strategies offer benefits to both institutional and retail investors.


Following an investment strategy typically involves two choices. First the investors must decide what asset classes to invest in - this may range from treasuries, equities, and real estate to precious metals. Very often, the choice is equities.

Once a decision is made on the particular asset allocation, the investor has to choose what type of strategy to utilize to implement that decision. This fund selection decision is made after the asset allocation decision. Viewing this fund selection decision using a behavioral finance perspective explains, in part, the popularity of both index strategies and the benefits of using an enhanced index approach.

Akey finding of recent research in behavioral finance is that when faced with investment decisions individuals do not make tradeoffs between return and risk as predicted by conventional finance. Rather than viewing return and risk in an objective sense, investors tend to trade off return and potential regret. Investors are more concerned with controlling regret than  minimizing risk, regardless of whether risk is measured by standard deviation, downside risk, beta, or some other statistic du jour. Regret is very different from risk. More specifically, regret is the pain that comes with the realization that a decision turns out, ex post facto, to be bad.

This is best illustrated with a simple example. Imagine that you are part of an office pool and have bet on the number 26 37 33 14 42 44 every week in the California State Lottery over the past six months. Your rigid adherence to this single number fails to produce any winnings and you decide to switch to 44 37 33 14 42 26. This decision does not affect the risk or expected return associated with buying the ticket so you decide to switch without informing your co-workers. Now suppose that the old number, 26 37 33 14 42 44, wins! Although your decision did not have any risk or return implications you realize that your decision does induce regret.

One way to avoid regret is to avoid choices - in the simple example above, staying with the original number minimizes the likelihood of regret. Similarly, when an investor chooses to invest in US stocks and implements the decision using an index fund, he or she is minimizing the potential regret from making a fund selection decision. Because the index fund, by definition, reflects the performance of the asset class, the investor who chooses this course of action never has to deal with the pain of regret as a direct result of the decision.

This 'zero regret' strategy of choosing an index fund, however, is not always optimal. Numerous studies by both academics and practitioners have demonstrated that equity markets are not perfectly efficient. Given there are opportunities to add value through security selection, we define the goal of an enhanced index strategy as maximizing return over a specified passive index (in our case the S&P 500) but at the same time minimizing the potential for regret. We minimize this potential for regret by controlling the economic exposure of our portfolio to mirror that of the index.

The portfolio, in our case typically containing 70 or 80 stocks, is managed to have sector exposure in 11 sectors defined by us that is nearly identical to the S&P 500 (within plus or minus one percent), and to have a market capitalization similar to that of the S&P 500. The average price/book of the portfolio, a measure traditionally used to measure when a portfolio has a 'value' or 'growth' tilt, is also constrained to match that of the index. Finally, only stocks that are in the S&P 500 are included in the investment universe, and the portfolio is constructed to have a standard deviation and beta that are similar to that of the index. The characteristics of such a portfolio on these dimensions are shown in Figures 1 and 2. Given the large number of constraints incorporated into the portfolio construction process and the resultant complications in designing the portfolio, a formal optimization process is required for the final portfolio. We use the BARRA optimizer.

Absent any positive (or negative) stock selection ability on the part of the manager, such a portfolio will perform like the S&P 500. In fact, in a perfectly efficient market such a portfolio should have an expected return identical to that of the S&P500 - by virtue of the fact that it is identical in virtually all the characteristics (like market capitalization, the ratio of book value to market value, and beta) that have been shown to be important in explaining the variability of equity returns across stocks. Potential regret, while greater than that from simply investing in an index fund, is quite low.

The possibility for regret is sometimes quantified as the 'tracking error' of the enhanced index portfolio versus the index. Tracking error is the standard deviation of the differences in return between a portfolio and the benchmark index - a good index fund therefore has a tracking error of close to zero. Active equity managers typically have tracking errors in excess of 6%. Those with tracking errors of less that 2% on an annual basis are generally classified as enhanced index strategies. Our definition of enhanced index strategies is somewhat less restrictive, as our constraints placed on the portfolio limit tracking error to the 3-4% range. While this is larger than that of a typical enhanced index manager, it is substantially less than that of the typical actively managed equity portfolio.


Having decided to construct a portfolio to 'control regret', we are faced with the challenge of also providing some enhancement over the passive index. Historically, investment managers have added value either by having access to superior information or by using public information to generate superior insights about prospective returns. Hundreds, if not thousands, of analysts place the typical company in the S&P 500 under a microscope on a daily basis. The marginal benefit of seeking out additional information is likely to be very small and arguably negative once the costs of seeking out such information is taken into account. Rather than engaging in a probably fruitless search for that extra nugget of information, we choose to focus our efforts on using publicly available information to generate superior valuation estimates.

Specifically, our approach is to use a dynamic valuation model that recognizes the time-varying nature of the relationship between fundamental characteristics and returns. Instead of focusing on the long-run relationship between a variable and returns, we focus on more recent relationships. In other words, we recognize the fact that investor preferences change as the business cycle evolves and the structure of the economy changes. For example, in the early stage of a recovery where growth is plentiful, investors are not willing to pay a premium for companies that have historically experienced higher growth rates. Similarly, in a rising-interest environment, investors pay relatively more attention to financial leverage as a variable. The tendency for particular styles of investment management approaches to go 'out of favor' for extended periods of time is also consistent with our view that the appropriate valuation model is dynamic and there does not exist a static 'all season' valuation model. The failure of some enhanced index strategies with a permanent 'value' or 'small-capitalization' tilt is also evidence of the cyclical nature of the payoffs associated with each of these characteristics.

Regression models provide a useful method by which to evaluate the importance of different factors. The monthly returns on each stock serve as the dependent variable, and the exposure to the characteristic in question serves as the independent variable. As depicted in Figure 3, the slope of the regression line - referred to as the payoff from taking exposure to that factor - is a measure of the strength of the relationship between each characteristic and returns. In order to compare payoffs across different factors, a stock's exposure is measured in terms of the differential in its value from the average value of the characteristic in the universe.

These differentials are expressed, very typically, in terms of standard deviations, so that 66% of stocks have an exposure between plus and minus one. Therefore, for example, a stock with an above average earnings yield (E/P) will have an E/P exposure greater than zero, whereas an expensive growth stock with a low E/Pwill have an exposure less than zero. In contrast, the earnings growth exposure of a value stock will be less than zero and the exposure of a growth stock on this same factor will be greater than zero. By definition, the average stock in the universe will have an exposure of 0.0 to all factors. The estimated factor payoff over a specified time period can then be measured as the incremental return from having an exposure of 1.0 to that factor. (Generally, exposures of greater than plus 2 or less than minus 2 are, by definition, not commonplace.) Note that this standardization is purely for convenience in that it allows direct comparison of the magnitude of the payoff associated with different characteristics of factors.

In Table 1 we show the average value of this payoff for six elected factors over selected time periods. These six factors were selected as being representative of the types of basic characteristics used in stock valuation. They encompass both measures of valuation and growth potential. Over the long-run time period (we use 1974-1998), the factor payoffs associated with historical earnings growth, measures of cheapness (earnings yield, book to price, and the ratio of operating cash flow to price), and profitability are, not surprisingly, all positive. The payoff associated with leverage is negative, indicating that companies with higher levels of financial leverage tended to have lower returns over this entire time period. The payoffs are stated in annual terms, so stocks with an exposure of +1.0 to earnings yield - what some would refer to as a value stock - would be expected to have had an annual average return 3% higher over this time period than the average stock.

  1974-98 1996-98 Recessions Expansions
EPSGrowth (1 Year) 1.24 1.56 1.50 1.17
E/P 3.02 2.26 2.01 3.28
B/P 2.09 (0.95) 3.26 0.18
C/P 0.85 2.75 (0.69) 1.21
Leverage (0.38) 0.58 (1.12) (0.19)
Return on Equity 0.74 1.32 0.40 0.82
Estimated using a universe of the largest 1000 stocks in the United States. Economic environment based on NBER definitions of expansionary and recessionary time periods.

The three years ending in 1998 are substantially different from the long run - witness the very high payoff to growth and profitability and the positive payoff even to financial leverage. Even more striking are the differential payoffs to the valuation measures - companies with higher than average levels of cash flow to price and earnings to price have had positive returns, whereas companies with higher levels of book to price have had negative returns. Even in terms of these six commonly used characteristics, these three in particular are very different from the long run.

Some insight into these differences can be drawn from evaluating these factor payoffs in different economic environments. Using the data published by the National Bureau of Economic Research, we classify each month as a recession or expansion and compute the average payoff in each of these differing economic environments. In a recession, earnings growth is scarce, so investors are willing to pay a premium for companies with strong recent growth. This is evidenced by a higher average payoff associated with earnings growth in a recession. The payoffs to E/Pand C/Pare both substantially lower in recessionary periods. The payoff to B/P, a measure of the value of the assets in place relative to the market value of the company, however, is substantially higher in a recession. In a recession, as earnings growth becomes scarce and stock prices fall, the value of the hard assets in place becomes an important determinant of stock price valuation. Naturally, highly leveraged companies do relatively poorly in a recession versus an expansion, as recessions are associated with declining earnings and higher default rates. There hasn't been a recession in a long time, but a similar analysis applied to the evolving expansion has produced good results.

The intent of this analysis is not to suggest that investors try to forecast the state of the economy, but rather that the relationship between each of these factors and stock returns are driven by changes in the economic environment. In the absence of shocks - such as the Arab oil embargo or the Asian crises -the economic environment does not change dramatically on a month to month basis. Thus higher (or lower) than average payoffs to a particular factor tend to be followed by similar higher (or lower) than average payoffs in the following month. In statistical terms, the payoffs to the factors are serially correlated. The magnitude of the correlation between the factor payoffs in successive months are shown in Table 2. Note that every one of the correlations is positive. This is clear support for our contention that payoffs to factors change, and that these changes are gradual and - this is the important point - to some extent predictable.

EPSGrowth (1 Year) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.090
E/P. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.118
B/P. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.070
C/P. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.080
Leverage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.102
Return on Equity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.140
Correlation estimated between monthly payoff in month t versus the payoff in month t-1

More evidence of the varying nature of factor returns are shown in Table 3. Here we show selected factor payoffs for several factors during the first nine months of 1999. For comparison purposes, the long-run payoff to these factors is also included in the table. This current year, 1999, has been different from the long run in many respects. The factor payoff to historical earnings growth has been negative whereas the payoff to forecast earnings growth has been positive. This indicates that investors have been placing increasing emphasis on forecast earnings growth and discounting historical earnings growth over the past year. The obvious implication is that forecast earnings growth should get greater weight over historical earnings growth in the valuation process. The payoff to earnings momentum has been sharply negative as is typical in an economic environment in which earnings growth is slowing. The returns to the valuation measures suggest that companies with high levels of cash flow to price have had higher returns, whereas those with high levels of earnings yield have had systematically lower returns. This may in part reflect the fact that operating cash flow is less subject to accounting gimmickry than net earnings. Consistent with the data shown in Table 2, the return to leverage as a factor has been significantly negative in 1999, as interest rates rise and earnings growth rates slow. In summary, in the current environment, as we write in the fall of 1999, it appears that forecast earnings is the preferred growth measure over historical earnings growth, and that cash flow to price is the preferred measure of cheapness. Companies with higher levels of leverage should have a much higher expected return in order to be viewed as attractive, as the market appears to be penalizing this factor more than its historical average.

  1974-98 1999 (Jan-Sep)
EPSGrowth (1 Year) 1.24 (4.44)
EPSGrowth (Forecast) 0.60 3.48
Earnings Momentum 1.68 (6.00)
E/P 3.02 (7.56)
C/P 0.85 3.12
Leverage (0.38) (1.08)
Estimated using a universe of the largest 1000 stocks in the United States

In practice, we evaluate the relationship between over 70 factors and stock returns on a monthly basis. In order to account for this positive correlation between payoffs in adjacent months, we weigh more recent data more heavily in determining the projected payoff on each of the 70 factors. The use of a large number of factors measuring relative valuation, earnings growth, profitability, risk, and liquidity allows us to reflect the complexity of stock valuation and capture changes in investor preferences for particular factors. In addition, the relationships between returns and all the factors are estimated simultaneously, taking into account both the volatility of each stock and the systematic variation of certain characteristics across industries. Making such econometric adjustments, while more time- and computer- intensive, improves the accuracy with which the payoffs to each factor are determined.

The combination of the expected payoff associated with each characteristic and each stock's exposure to each of these same characteristics is easily translated to expected returns on each stock. These stock-specific 'alphas' are then translated into a specific portfolio by using an optimizer to build a portfolio subject to our regret minimization constraints.

One critique of this dynamic valuation approach is that it is reactive and not anticipatory. However, we do not believe that we can 'expect the unexpected.' When there is shock to the economy, our expectations are that the portfolio, by virtue of its construction, will respond the same way as the unmanaged S&P 500 index. Investors in enhanced index strategies should draw comfort from the fact that the potential regret which can be incurred by an investor in this strategy is not dependent on the managers' ability to forecast shocks to the economy.


Prior to launching the strategy in October 1996, we simulated its performance over the period January 1991 through September 1996. Over this time period the strategy outperformed the S&P500 by over four percentage points on annualized basis. Since the inception of live track record, the strategy has continued to outperform the S&P500 index, sometimes by over seven points on an annualized basis. As a primary objective of the strategy is to control regret, we show in Figure 4 the difference between the returns of the strategy and the benchmark over this simulated period. The periods of underperformance are short and limited in terms of their magnitude, as would be expected given the portfolio construction. There is no meaningful difference between the actual (post October, 1996) and simulated (pre-October, 1996) performance in terms of the risk of the portfolio and its tracking error versus the S&P500 index.

Such an enhanced index strategy has obvious benefits over a more traditional active approach. First, the degree of underperformance over a shorter horizon is managed so as to limit the potential regret an investor may have in choosing such a strategy. Second, the valuation model adapts to reflect the economic environment so that the investor does not have to make a decision to maintain or terminate a manager relationship whose particular style is 'not in favor.'

Against a zero regret strategy of investing in the index, however, the benefits of using such an approach will be determined purely by the alpha of the strategy versus the potential regret which can be induced. For dogmatic believers in perfectly efficient markets, the expected gains from enhanced indexing are zero, and possibly negative after transaction costs and fees are included. However, if there exists even slight levels of inefficiency, enhanced index strategies such as we describe are an attractive alternative to indexing. The magnitude of potential outperformance - in the 3%-6% range on an annual basis without taking on any additional risk - is comparable to the historical differential premium of stocks over bonds. Surely such a level of outperformance justifies the potential regret?

Find your next ETF

Reset All