Out of Style

April 01, 2000

Many investment managers follow a different list of stocks than those included in well-known style indexes, leading to disparities in performance between style indexes and manager universes. Managers and fund sponsors need to be aware of these differences in building portfolios and structuring teams of investment managers.

In a recent working paper, Caveat Emptor, Richards & Tierney ("R&T") examined (1) the divergence in the performance of investment styles and (2) the narrowness of the market.

Recent investment style performance has been the most divergent that R&T has observed, both between and within investment style categories.i By investment styles we refer to the major categories of large-cap growth, large-cap value, small-cap growth and small-cap value. R&T builds and maintains investment style indexes going back to 1976.

Also, the recent performance of the domestic equity market has been extremely narrow, meaning that a small number of stocks account for the market's overall performance.

As a result of these two phenomena, the Caveat Emptor paper issued a warning to fund sponsors to be very careful in evaluating investment managers' recent performance. The purpose of this current paper is to further examine this issue of performance evaluation, particularly in this recent market environment.

Over the years, there has been a growing awareness that a manager's performance should be evaluated relative to an appropriate benchmark that captures the manager's investment style. One alternative that many fund sponsors and consultants have adopted is the use of generic investment style indexes. These indexes were introduced to the investment community in the late 80's and are intended to capture broad (value, growth, large, small) characteristics of the market. Since the performance divergence that we have observed was not only between investment style categories, but also within investment style categories, we want to examine the materiality of these differences with respect to performance evaluation and portfolio management.

Since there are a variety of consulting firms who produce investment style indexes, each with a different definition of investment style, style index returns can be materially different. In addition, a money manager's definition of the firm's investment philosophy and style can be significantly different from that of an outside consultant.

Amuch better alternative to an investment style index is a manager custom benchmark. This is the universe of securities that are actively researched by the investment manager weighted to reflect the normative risk characteristics of the manager's active portfolios. The primary difference between a custom benchmark and a generic style index is that a custom benchmark is unique to each manager while the generic style index is meant to cover a large number of managers that are pigeonholed into a particular investment style category. As a result, there is a difference in security composition and weights between a generic style index and a custom benchmark. The question we consider in this paper is the materiality of these differences, particularly in the recent narrow market. Although it is a case study and not definitive, we believe it illustrates the performance evaluation risks and portfolio management problems that confront fund sponsors and investment managers.


We looked at the breadth (or conversely, the narrowness) of the total domestic equity market over the past several years. We examined the effect of the highest impact securities on overall market returns in recent years versus in the past. Our analysis was based on the R&T 2500ii index (hereafter dubbed the "market"). This is a market capitalization-weighted index comprised of the largest 2,500 stocks that are traded by institutional domestic equity managers.

Akey concept to help us understand the narrowness of the market is the performance impact of each stock in the market. The performance impact of a stock is the stock's weight times the difference between its return and the market return. For example, Microsoft's rate of return in 1999 was 68.36%, and the return of the R&T 2500 was 20.18%. Since Microsoft's weight was 2.68%, the performance impact of Microsoft on the R&T2500 index in 1999 was

(2.68% * (68.36% - 20.18%)) = 1.29%

In this fashion we calculated the performance impacts of all the stocks in the R&T 2500. We then sorted the securities beginning with those stocks having the greatest positive impact. We then eliminated each of the top contributors and recalculated the index return after each stock was eliminated.

The following table (Table 1) shows 1999 results. As you can see, Microsoft had the largest positive impact on the market's 1999 return. Since Microsoft is the largest stock in the index and it had a large return in 1999, it is not surprising that it had the most positive impact on the market's return. Qualcomm, however, ranked 445 according to its size in the R&T 2500 index at the beginning of 1999, yet it had the third largest impact on the market's return. This is because its return was so extraordinarily high relative to other security returns (2,619.42%).

Positive Impact Rank Security Name Beginning Weight Security Annual ROR Market Annual Annual ROR Excluding…
        20.18 %
1 Microsoft 2.68 % 68.36 % 18.86
2 Cisco Systems 1.04 130.84 17.64
3 Qualcomm 0.03 2619.42 16.72
4 General Electric 2.63 53.54 15.69
5 Oracle 0.30 289.78 14.82
6 Nortel Networks 0.27 305.29 13.97
7 Walmart Stores 1.50 70.42 13.05
8 Sun Microsystems 0.25 261.75 12.37
9 Citigroup 1.02 70.10 11.71
10 E M C Corp 0.32 157.06 11.19

As shown in Table 1, the performance of the index without Microsoft would have been 18.86%, not 20.18%. If we eliminate the ten top impact stocks from the R&T 2500 index, the 1999 rate of return drops from 20.18% to 11.19%. These ten stocks account for nearly one half of the 1999 return. If we eliminate the 50 stocks with the largest positive impact on the index's performance, the return is 1.39%. In 1998, a similar phenomenon occurred. The rate of return of the R&T 2500 index was 25.00% in 1998. Again, if we eliminate the top ten stocks having the largest positive impact, the return is reduced to 16.85%; if we eliminate the top 50 stocks, the return falls to 7.38%.

 To put this in historical perspective, we performed the same analysis for all years between 1976 and 1999. For each year, we removed the 50 largest contributing stocks from the index and calculated the performance of the index without them. We plot the results below. As you can see, the slopes of the declines in returns were much more sudden and sharp in 1998 and 1999 than the slopes in prior years.

In all years between 1976 and 1997, the impacts of individual stocks were much smaller. In fact, removing the top 50 securities in prior years would have resulted in an average decrease in performance of only 5.29%. In 1999, the decrease in performance would have been 15.64%. In 1998, it would have been 14.09%. Thus, the exclusion of these key securities from a portfolio would have had very dramatic performance impacts in 1999 and 1998. The above analysis illustrates that the vast majority of the 20%+ rates of return can be attributed to only a few key stocks in 1998 and 1999. These were the narrowest markets in the 24 years for which we have data.

On the flip side, we explored the impacts of the worst performing securities. Although the case was not as extreme, we found that the bottom end of the market was also highly concentrated in 1999, but not in 1998.


Now let us consider a case study that illustrates how this market phenomenon has affected the performance evaluation of an investment manager. Investment Manager ABC is a large "value" manager. Manager ABC's investment philosophy is to invest in securities that have certain dividend yield and business characteristics. As a result, the firm's research and portfolio selection focuses on a unique universe of stocks. This universe consists of approximately 300 stocks and evolves slowly over time as a result of changes in companies' dividend and business characteristics. Since the firm is interested in closely monitoring its investment process and evaluating the firm's analysts and portfolio managers, a custom benchmark is built and maintained. It is built by weighting the universe of stocks based on the long-term normative risk characteristics of Manager ABC's actual portfolios. Interestingly, the stocks in the benchmark are thus neither capitalization weighted nor equally weighted.

In 1999, the rate of return of Manager ABC's custom benchmark was 1.96%. The rates of return of two well-known large "value" style indexes were much greater. The Russell 1000 Value index had a return of 7.35% and the S&P Large Value index had a return of 12.72% in 1999. Since Manager ABC's actual portfolios had returns in the 2%-3% range, the question is "Did Manager ABC perform well or poorly in 1999?" Since Manager ABC's actual portfolios are selected from a unique universe and Manager ABC is constrained to the opportunities provided by that universe, we can assert that the Manager ABC performed well in a difficult environment. Perhaps, a more interesting question is why the returns of Manager ABC's custom benchmark and the style indexes are so different.

The obvious answer is that they are different portfolios with different securities and weights. Since all three portfolios have similar risk characteristics, the primary explanation for the return differentials is the individual security composition of each. The generic style indexes had a different list of securities and weights than Manager ABC's custom benchmark. In fact, there were a large number of securities in each style index about which Manager ABC did no research and held no opinion.iii

In the Venn diagram, the circle labeled "I" represents the securities in a generic style index, e.g. S&PValue. The circle labeled "U" represents the universe of securities in Manager ABC's custom benchmark. The "A" area represents the actual portfolio, which is a subset of "U". The actual portfolio represents the most attractive investment opportunities within the research universe as determined by Manager ABC. The intersection of "U" and "I" is the list of securities in the style index about which Manager ABC holds an investment opinion. However, much of index "I" is outside "U." These are securities in the index about which Manager ABC holds no opinion. We believe that Manager ABC should not be held accountable for the performance of these stocks. (Note also that some stocks followed by Manager ABC lie outside index "I", meaning those stocks do not qualify for the generic index.)

With respect to Manager ABC, we measured the magnitude of the "I-U" area by identifying the securities in a generic investment style index but not in Manager ABC's universe. We performed this analysis for the S&P Value and Russell 1000 Value indexes over the past 12 years.

Chart 2 above illustrates these coverage ratios. This chart shows that anywhere between 25% and 50% of the stocks in the two generic style indexes were not being actively researched by Manager ABC and that the number has trended upward over the years.

Now we need to consider the performance of these stocks that are in the generic style indexes but not in Manager ABC's universe ("I-U"). In addition, we need to compare the performance of these stocks with that of Manager ABC's custom benchmark (weighted universe list). For example, during 1999 the return of those securities in the S&PValue index but not in Manager ABC's universe was 27.65%. Since Manager ABC's custom benchmark had a return of 1.96% in 1999, the difference was 25.19%.iv This means that Manager ABC was not researching approximately 40% of the S&PValue index and the performance of these stocks was 25.19% greater than Manager ABC's custom benchmark in 1999.v

Although this difference seems to be large, we need to see how it compares with other years. Chart 3 below shows these differences from 1988 to 1999.vi The chart shows the differences for both the S&PValue and the Russell 1000 Value. We can see that in the 1988-1990 period the performance differentials were negative because the custom benchmark outperformed, an outcome that probably would not bother money managers or clients. In the 1991-1993 period the performance differential was positive in the range of 5%. Although this was not a particularly good outcome for the manager or client, it occurred at a time when generic style indexes were just beginning to become popular and were not widely used as performance standards. Also, the performance differences between the manager's custom benchmark and the generic style indexes were not that great. In the 1994-1998 period the performance differentials were minimal. However, as noted earlier, in 1999 the performance differential was very large (about three times greater than previously observed). In addition, the coverage ratio was lower. The (1) large performance differential among securities and (2) low coverage explains why the manager's active portfolio and custom benchmark materially underper-formed the two generic style indexes.


Recall the earlier discussion of the narrowness of the market and how the return of the R&T 2500 dropped precipitously in 1999 if only a few key securities had been removed from the market portfolio. What would have happened to the value style indexes if only a few securities had been removed? We performed the same analysis on the style indexes.

  S&P Value   Frank Russell 1000 Value
  Portfolio Returns Returns Without Top 20 Number of Top 20 Notin Manager Universe Portfolio Returns Returns Without Top 20 Number of Top 20 Not in Manager Universe
1st qtr 2.79 0.00 16 5.81 -1.67 12
2nd qtr 10.79 7.47 8 11.30 7.59 6
3rd qtr -9.25 -12.50 15 -7.76 -12.48 10
4th qtr 8.23 1.19 18 5.61 0.72 14

Table 2 above lists the results. As an example, the rate of return on the S&PValue was 8.23% during the 4th quarter 1999. Without the top twenty positive impact securities, it would have been 1.19%. Interestingly, of the top 20 securities, 18 were not researched by Manager ABC. See Table 3.

Impact   9/30/99 Quarter Return
Rank Security Name Weight Return Excluding…
1 NORTEL NETWORKS CORP 1.39 98.11 6.97
2 MOTOROLA INC 1.09 67.47 6.29
3 CITIGROUP INC 3.04 26.89 5.63
4 MRGN STNLY DN WTR DISC 1.04 60.40 5.02
5 AMERICAN INTL GROUP 2.76 24.43 4.43
6 HEWLETT PACKARD CO 1.88 25.52 3.98
7 VIACOM INC NON-VTG-B 0.60 43.05 3.71
8 CBS CORP 0.67 38.24 3.45
9 SPRINT CORP PCS COM 0.67 37.47 3.19
10 AMERICAN EXPRESS CO 1.24 23.33 2.89
11 SPRINT CORP 0.96 24.31 2.65
12 * ALCOA INC 0.47 34.15 2.48
13 NOVELL INC 0.14 93.05 2.32
14 * MARSH & MCLENNAN COS 0.37 40.48 2.15
15 GENERAL INSTR CORP DEL 0.17 77.08 2.00
16 3COM CORP 0.21 63.48 1.85
17 COMCAST CORP CL-A SPL 0.61 26.80 1.66
18 APPLE COMPUTER INC 0.21 62.39 1.51
19 CENDANT CORP 0.26 49.65 1.36
20 ELECTRONIC DATA SYS NW 0.53 26.74 1.19

Of the above 20 stocks, Manager ABC researched only Alcoa and Marsh & McLennan. Therefore, Manager ABC had no opinion regarding Nortel, Motorola, Citigroup, Hewlett Packard, etc. We believe Manager ABC should not be held accountable for the performance of these non-researched stocks.

With respect to the Frank Russell 1000 Value, the returns without the top impact securities declined most in the first quarter of 1999. During this quarter, Manager ABC held opinions on only 8 of the top 20 securities. He did not hold opinions about Motorola, Viacom, Hewlett Packard, Sprint, and Apple Computer, which were in the Russell 1000 Value and were some of the top impact securities. Also, it is interesting to note that Qualcomm was in at least one generic value index during the first half of 1999, a company that is the antithesis of what Manager ABC would invest in.

In prior years these coverage issues have existed. However, in broader markets, the impacts have not been as noticeable. In these cases, Manager ABC, clients and consultants may not have noticed the inappropriate comparison to a style index. The narrowness of the market in 1999 caused these issues to be much more significant. We would contend that the same forces that shaped the market in 1999 were still evident in the market in early 2000. In a paper recently published by Sanford Bernstein & Co.,

"Value Index Distortion," it was asserted that the construction rules for generic style indexes might be causing the indexes to include inappropriate securities. In particular, the rule of dividing the world in half according to capitalization weight causes many securities to be included in the generic value indexes which are not in a typical value manager's universe. Examples cited were Motorola, Nortel Networks, American International Group, among others. All those securities are listed on the top twenty impacts (Table 5) above. They had significant impacts on the returns of the value index, and one might argue that they are not really value securities.


We have no issue with how S&Pand The Russell Company define and build their investment style indexes. We believe that there is no one "right" definition of a generic investment style index. We have examined the Russell 1000 Value and the S&P Value, because they are popular generic indexes. There are also a variety of other consultants and investment research firms who produce generic style indexes, all with different construction rules, and we have no issue with them either. Likewise, we have no issue with how investment managers define their firms' investment style. Investment managers, whether they are "value," "growth," "large," or "small," all define their universes (investment styles) differently. Our objective is not to argue the "rightness" or "wrongness" of any index, but only to underscore the importance of the differences between all of these generic style indexes and manager's investment styles as reflected in their research universes.

We believe investment managers should be held accountable. However, the accountability standard should be comprised of securities for which money managers have informed investment opinions. We believe money managers are in the best position to define these accountability standards. Further, they should be defined in advance and made available to all interested parties.

The implications of this paradigm for fund sponsors and investment managers are improved definitions of investment risk. In particular, we are referring to active management risk and investment style risk (misfit risk). With respect to active management risk, we have eliminated the "noise" related to securities that are not researched by the manager. With respect to investment style risk, we believe a custom benchmark will better define the investment manager's world and area of expertise. As a result, fund sponsors are in a much better position to structure teams of investment managers and accomplish their investment objective of outperforming a market target. In addition, we believe a much stronger covenant between investment managers and fund sponsors will exist. The objective of this paper was to provide an illustration of potential performance evaluation problems, in light of the recent market environment. It was clear to us that comparing a manager's performance with the overall market performance would be dangerous. However, it was not clear whether the same would be true with a generic style index, so we chose to undertake a case study.

There are a number of issues that were not addressed in this paper. For example, we did not examine the impact of securities that are in the manager's universe but not in the generic style index. We also did not consider the impact of differences in the weights assigned to the securities that were common to the manager's custom benchmark and the generic style indexes. In this case study, comparison to a generic style index was a problem for 1999 but not necessarily for 1998. Other investment managers with other research universes may have experienced different results. Therefore we believe each manager must be examined individually.

The long-run success of a pension fund depends far more on the design and consistent implementation of sound investment policy than it does on such ephemeral activities as manager selection. In fact the hiring and firing of managers is very costly, and we believe this activity should be minimized. Good benchmarks allow plan sponsors to truly understand the performance of their investment managers and help them avoid the costs of firing and replacing their managers unnecessarily. These benefits increase the probability of a successful investment program.



i On average, the difference between large value and large growth has been around 13% per year. The maximum difference occurred in 1999 (55.75%) and was over four times as great as the average. Likewise, the average difference between small value and small growth indexes has been around 15%. The maximum difference also occurred in 1999 (60.63%) and was also over four times as great. In both of these cases, the value portfolios significantly underperformed their growth counterparts. Therefore, if a portfolio had a "slight" value orientation relative to a benchmark or market index, the resulting underperformance during 1999 would have been four times as great as expected based upon historical experience. In 1998, the difference between small growth and large growth was 39.14%, almost three times the historical average. In this case, any bias towards small-cap securities would have resulted in a performance differential three times as great as normally expected.

Styles Compared Average Maximum Minimum
Large Value vs. Large Growth 12.84 % 55.75 % 0.51 %
Small Value vs. Small Growth 15.15 60.63 0.97
Large Value vs. Small Value 9.13 20.39 0.93
Large Growth vs. Small Growth 13.48 39.14 2.14

Index 1998 returns 1999 returns
R&T Large Value 7.99 -11.63
S&P / Barra Value 14.67 12.72
Frank Russell 1000 Value 15.63 7.3

iiThe R&T 2500 index is rebalanced quarterly. For this study we considered only the securities that were in the index at the beginning of the year and also at the end of the year. The actual returns of the R&T 2500 index include the returns of security additions and deletions throughout the year. Typically, the difference between the actual R&T 2500 annual return andthe R&T2500 that includes only the securities that survived the full year averages about 36 basis points. However, in 1999 the difference was 251 basis points. This is because of numerous mergers and acquisitions in 1999.

iiiWe believe managers should choose their investment universes based on their philosophy and perceived sphere of comparative advantage rather than simply adopting an externally constructed list of securities. ivWe used geometric return relatives as a means of comparison. For example, the 25.19% difference is equal to ((1.2765/1.0196) - 1) * 100. vIf we were to compare him to the S&P Value index, we may (incorrectly) decide to change the manager. However, if we were to compare him to his own benchmark, we would recognize the value added and instead focus on broader issues such as manager structuring. viAdditional data are shown in the table below:

  Universe Frank Russell 1000 Value   S&P Value  
Year U I I - U I - U vs U I I - U I - U vs U
1988 27.78 23.16 22.88 -3.84 21.67 19.01 -6.86
1989 28.01 25.19 22.65 -4.19 26.13 24.15 -3.02
1990 -7.49 -8.08 -12.41 -5.32 -6.85 -13.70 -6.71
1991 28.70 24.62 31.81 2.41 22.56 27.03 -1.30
1992 12.69 13.81 18.54 5.20 10.52 15.31 2.33
1993 13.48 18.12 22.93 8.33 18.59 23.54 8.86
1994 2.53 -1.99 -1.85 -4.28 -0.63 0.74 -1.75
1995 40.08 38.35 34.42 -4.04 37.00 33.25 -4.87
1996 23.19 21.64 24.97 1.44 21.99 25.63 1.98
1997 31.88 35.18 33.89 1.52 29.98 25.90 -4.54
1998 13.86 15.63 13.78 -0.07 14.67 15.04 1.03
1999 1.96 7.35 23.33 20.96 12.72 27.65 25.19

Find your next ETF