Perfection Impossible

April 01, 2002

None of us is as smart as all of us

- Anonymous quote hanging in the office of James Vertin, Head of Wells Fargo Management Sciences Department and backer of the first index fund. - circa 1971

I. Introduction

Indexes have advanced tremendously since the debut of Charles Henry Dow's pioneering average in the late nineteenth century. Indeed, particularly in the years since the launch of the first index fund in 1971, indexes and the art and science of indexation have risen to meet an ever-growing demand of uses, products and indexed assets. As the theme of this issue of The Journal of Indexes is "in pursuit of the perfect index," it is important to note that continual improvement in index methodology benefits investment managers and their clients alike, and will undoubtedly continue to do so in the years ahead.

Our industry should, of course, pursue better index methodology, but I believe we should not obsess over what inevitably would be a quixotic quest for one perfect index or index methodology. Investors use indexes for a diversity of purposes, and the needs of investors and the market evolve dynamically over time. Indexes must reflect this evolution and diversity. Competition among index providers and index managers will take care of the rest, ensuring that continual improvement occurs, and that investors have an optimal choice of indexes and index-based products.

The perfect index is in many ways "the impossible dream," but like Don Quixote, our pursuit of the ideal can make the world of indexes a much better place. This essay discusses the uses of indexes, and defines the characteristics of a good index as well as covering the critical tradeoffs in benchmark design. Finally we discuss how you go about choosing an index to suit your purposes. An index that is 'perfect' for one investor might be completely inappropriate for another investor. Thus when striving for perfection, one must continuously ask the question "perfect for what use?"

II. How Indexes Are Used - the Four Fundamental Uses

There are four fundamental uses of indexes, which I describe below. This article focuses on equity indexes, but most of the uses, the criteria for a good equity index and the tradeoffs necessary in equity index investment are similar for other asset classes.

Accurate Gauge of "The Market" and Investor Sentiment

Since their inception, market indexes have been widely used to answer the question, "What is happening in the world at this minute?" Condensing the prices of diverse securities in a market to a single statistic is useful because it reveals the net effect of all factors at work in a market. These include not only hopes and fears specific to companies in the index, but also broader factors - war, peace, economic expansion, recession, and so forth - that can potentially have an impact on share values. Thus a frequently updated stock-market index gives an indication of a country's economic outlook. Following the September 11th 2001 terrorist attacks in New York and Washington, for example, the relatively short duration of the market drop was a significant boost to the nation's confidence.

Asset allocation

As asset allocation has become accepted as central to sound investing, analysts have studied the historical returns and other characteristics of indexes in an attempt to understand the behavior of the asset classes they represent. An index constructed on a consistent basis across time allows one to calculate long-run rates of return, the risk of different asset classes, and the changes in risk of a given asset class over time. Indexes can also be compared to calculate correlations and gains from diversification across asset classes, and to perform other analysis relevant to developing an investment strategy.

Performance measurement

One of the attractions of having a market index available is that it answers the question, "Did I beat the market?" The natural human desire to best one's competitors motivates investors to compare their portfolio returns to index returns. The modern science of performance measurement, evaluation and attribution takes this obsession to the next level. It draws on the academic achievements of the 1960s - the capital-asset pricing model and related work - in determining to what extent, and why, a particular portfolio beat or was beaten by a market index.

Basis for Investment Vehicles

With the advent of the capital-asset pricing model, and other theories suggesting it is difficult to beat the market on a risk-adjusted basis, market-capitalization weighted indexes turned out to be well-suited for an important and revolutionary new use: index funds . By simply matching the holdings of a well-constructed index, a portfolio manager can provide the return of the index, net of expenses. Moreover, many active investors - particularly quantitative active, risk-controlled, and enhanced-index managers - use the contents of an index as their starting point, deviating from index weights according to the degree of conviction they have that a particular stock is more or less attractive than the market as a whole.

III. The Seven Key Criteria of a good index

Indexes are useful as benchmarks for active management, as the basis for index funds, and as proxies for asset classes in asset allocation. Ideally, one should choose an index that can be used for all three purposes simultaneously, as added fungibility makes the utility of the benchmark that much greater. When selecting indexes to use for one or more of these purposes, one must consider all of their characteristics and determine which indexes best fit the investor's needs. Still, no equity index is perfect, as tradeoffs are involved, and these are discussed in Section IV.

How should one choose from among the competing alternatives? In addition to market-capitalization weighting, which is a prerequisite of a good index and is common to all indexes covered here, there are seven key criteria that are useful in identifying a good broad-capitalization equity benchmark. Obviously there are many more minor criteria, but we find it useful to categorize the major criteria in 7 broad groupings. These criteria formed the basis of the "Benchmarks 101" article published in the Journal of Indexes in the second quarter of 2001 as well as earlier essays published by BGI. Please visit to read the Q2 '01 article, as well as a list of additional criteria, under the heading of "Don't Stop at Seven." 

The Seven Key Criteria of a Good Broad-Capitalization Index

1. Completeness

Does the index accurately reflect the overall investment opportunity set, both in terms of market cap-range/country coverage and company inclusion? The more complete an index - the broader and deeper its coverage - the more effectively it represents the investable universe for both active and index managers. By spreading its allocation among most of the available securities and markets, a comprehensive index maximizes diversification. Completeness is probably the most important of the 7 key criteria, as complete coverage of the targeted asset class is the foundation for the utility of indexes in all of their potential applications.

2. Investability

Does the index include only those securities that can be effectively purchased by investors? For non-U.S. benchmarks, does the index screen out shares and market segments that are restricted for foreign investors? Obviously the goal of investability stands in juxtoposition with the objective of completeness, and the tradeoff between the two often requires a user to make an explicit preference decision.

3. Clear, published rules and open governance structure

How transparent are the rules that govern the benchmark? Are these rules well-established and publicly available? Such rules provide predictability to both portfolio managers and asset owners, and make it easier to anticipate how changing market conditions will be reflected in the benchmark. For an index to be truly useful to the various users of benchmarks, index construction rules should be fully transparent, especially during index reconstitution periods and during major corporate actions.

4. Accurate and complete data

For an index to be useful, return and constituent data must be accurate, complete and readily available. Investors should have access to at least the following information: price/total/net dividend returns, consistent sub-indexes, quality and timely release of data, transparent release of index changes, and historical returns. While some believe that the ready availability of index data somehow hurts investors in index-based products, I believe the opposite is true. The more understanding of the methodology and constitutents of an index, the more comfort investors have in products based on the index.

5. Acceptance by investors

In general, investors prefer an index that is well known and widely used. This gives an investor faith in the ongoing integrity of the index, since it will be under scrutiny from a variety of market participants. Furthermore, wide use enables effective peer group comparison. The performance of non-standard indexes and index products are invariably compared to the standard benchmark, Academic and proprietary research, the basis for asset allocation studies, tends to focus on established benchmarks to provide relevant insights for investors, Finally, without broad acceptance of an index, there might be inadequate availability of supporting investment products based on the benchmarks (including active funds and derivative products).

6. Availability of crossing opportunities, derivatives, and other tradeable products

Indexes that are widely used, especially within pooled investment vehicles, offer potential cost savings because they provide crossing opportunities within the fund complexes of large institutional investment managers. Crossing allows an institutional investment manager to equitably match buy and sell orders without the typical costs that would be incurred in the open market. Such indexes also generally create a more liquid/cheaper to trade OTC derivatives market, particularly in total-return swaps. The availability of listed futures/options on some major benchmarks, and the proliferation of ETFs on virtually all major benchmarks/asset classes, further benefits asset owners and portfolio managers who use these accepted benchmarks. Ideally, a widely-used benchmark fosters a virtuous circle of activity by a critical mass of investors, in turn creating the potential for crossing trades/activity. Broad acceptance of a benchmark creates a "network effect" between fund managers, sell-side brokers and the cash and derivative markets that reduces transaction costs for movement in and out of index portfolios.

7. Relatively low turnover and related transaction costs

All indexes incur a certain amount of turnover as they maintain index constituents in line with their stated methodology. In general, the lower the turnover, the fewer rebalancing costs are incurred, and the easier the index is to track. By design, a broader benchmark favors lower turnover, while an index that works within a narrowly defined market segment has greater turnover and transaction-related costs. Furthermore, and index with a pre-defined number of stocks (e.g. S&P 500, Russell 2000, S&P Latin American 40, etc.) will have some degree of additional turnover to maintain the fixed number of constituents.

BGI and our primary competitors can (and regularly do) provide detailed analysis of the major U.S. and international/global indexes for our clients along the lines of these and other criteria. Instead of assessing the specific indexes in the limited space of the Journal, I would urge investors to consult with us or one of our esteemed competitors for the full analysis. When the "What benchmark should I use" question is posed, it is inevitable that it will be answered with another question: "What purpose will the benchmark be used for?" The discussion will then turn to the tradeoffs inherent in any benchmark decision -- does the client want a highly-liquid tradeable product for tactical allocation purposes or the maximum coverage of an asset class for strategic asset/liability modeling? On the other hand, perhaps a compromise can somehow accommodate both needs. Finally, some investors have such unique requirements (social screens, tax consequences, completion portfolios, etc) that a custom index is required; obviously no "one size fits all" index would be appropriate in these cases! .These sorts of tradeoffs and compromises permeate the decision process. It is critical to understand not only what you gain, but also what you give up in choosing a particular index.

IV. The Five Inherent Tradeoffs Involved In Index Construction And Selection

In constructing a good index, there are always tradeoffs in both methodology and implementation. These tradeoffs are critical to gaining an understanding of why the one-size-fits-all perfect index cannot exist.

The Five Inherent Tradeoffs of Index Construction

1. Completeness vs. Investability

From a purely theoretical standpoint, the ideal index includes every security in its asset class. Few investors know exactly how many stocks are listed in the United States, but the Wilshire 5000, so named because it was originally composed of 5000 stocks, contains over 7,000 stocks and thus includes more issues than any other widely-distributed U.S. equity index. However, many of the smaller stocks in the Wilshire 5000 are illiquid, and investors have a difficult time trading them. For these reasons, a somewhat less broad index is more "investable" and accessible. No full-replication index fund has ever been constructed for the Wilshire 5000, nor should it be. Similarly, there are well over 12,000 listed stocks in the total non-U.S. equity universe, and over 5,000 in the developed international universe.

2. Reconstitution and Rebalancing Frequency vs. Turnover

Reconstitution, which is the process of periodically deciding which stocks meet the criteria for inclusion in an index, is a source of turnover (which is costly to investors) because the manager must trade to keep pace with changes in the index. However, because timely reconstitution and rebalancing (which is the process of adjusting the weights of stocks in the index for changes in the number of shares outstanding) is what enables an index to track accurately the asset class it is designed to represent, there is a tradeoff between such accuracy and trading costs.

Reconstitution-related transaction costs are primarily a burden for small- and mid-cap indexes, and for style-specific indexes, all of which fall outside the scope of this article. In those indexes, companies with large weights in the index frequently cross the boundary that qualifies them for inclusion. Broad-cap indexes, in contrast, mostly experience turnover in their smallest-cap stocks, making turnover less of a problem when measured by the weight in the index of the stocks being traded. Nonetheless, turnover is costly whatever its source or frequency , and a cost advantage accrues to indexes that have less of it.

In terms of reconstitution-related turnover and trading costs, indexes that have no fixed limit on the number of stocks, and that are all-inclusive in terms of their capitalization range, have a small but notable advantage over indexes with a fixed number of stocks. This is because an all-inclusive index generally gains or loses stocks only because of new listings, delistings, and other changes in the identity of the stocks in the market or the market's industry composition.

3. Precise Float-Adjustment vs. Transaction Costs

As float-adjustment gains momentum and acceptance, the question of how to apply free-float to benchmarks has gained prominence. While this may seem relatively arcane, there is a significant potential cost impact of index providers 'going too far' toward incorporating precise float-adjustment (which is difficult to measure precisely anyhow). This in fact could be a case where "perfect" float adjustment is actually worse for users than simply 'adequate' or representative adustment,.

4. Potential "Index Effect" vs. Liquidity/ Crossing Opportunities

The general tendency for a rise in the price of a stock bound for inclusion in an index and the fall of one that is to be dropped has been well documented. When an index is widely used, the price impact is bound to be greater, but so are the crossing-opportunities and the liquidity of index constituents, which can help mitigate the market impact cost of rebalancing and reconstitution. Furthermore, it is possibly the case that the long-term price premium accorded to index membership can also compensate for the occasional - but unpredictable - deadweight cost of the "index effect"

5. Objective and Transparent Rules vs. Flexible Judgment-based methodology

Some broad-cap equity indexes are constructed using rules that are either rigidly or generally objective, while others are constructed using judgment. The advantage of objective rules is that any investor with access to the relevant data can predict, more or less accurately, what stocks will be added to and deleted from the index. This enables investors to trade in anticipation of (rather than in reaction to) additions and deletions, and in general to manage the index replication process in an orderly and efficient manner. Active managers also find it useful to be able to predict what will be in the index to which they are benchmarked.

The use of judgment in selecting stocks for an index, however, enables the index constructor to achieve certain traits that cannot be achieved with objective rules, and that constructors of judgment-based indexes claim are desirable. Standard and Poor's, which uses judgment in selecting stocks for its S&P 500 and S&P 1500 indexes, asserts that its indexes are superior in terms of stability, low turnover, and accurate representation of the real economy. The S&P indexes can achieve these traits specifically because the index construction staff need not act mechanically in selecting and removing stocks, and can take conscious steps to construct an index with the desired characteristics. Similarly, MSCI tends to use more judgement in the implementation of its major index changes, often in an effort to minimize turnover, and they increasingly consult the industry for feedback.

V. Would Proposed Indexation "Solutions" Actually Distort the Core Role of Benchmark Indexes?

As would befit a dynamic and innovative industry, there are a variety of proposed "solutions" to the shortcomings of indexes that have been introduced in past year or so. The problems they attempt to address generally focus on the transaction costs and market impact of index changes, and thus the solutions are rooted in the belief that a change in approach would reduce that impact/cost as well as providing other benefits. The "solutions" that have been put forward can be grouped in three broad categories:

  - Introduction of funds based on "silent" or proprietary indexes.
- Switch to peer-based/average manager holdings-based benchmark
- Adaptation of a "pure passive" approach to index portfolio management

Response to The Concept of the "Silent Index"

Gary Gastineau of Nuveen Investments proposes limiting index transparency to minimize index effect. The basic tenet of efficient markets is transparent information flow and equal access to information source. Confidentiality of trading plans and opaqueness in index methodology creates inequity in information flows and increases uncertainty. Uncertainty raises risk and volatility, and higher risk and volatility increase trading costs and market impact, bringing us back around to the issue we wanted to eliminate. Recent research by Simon Hookway of Westbury Asset Management has in fact demonstrated that pre-announced constituent changes have a substantially lower "index effect" , For exchange-traded funds, which Gastineau is specifically addressing, transparency is not only an important feature of the ETF product, but also an integral part of the ETF mechanism. It lowers the cost to the investor because it enables precise hedging by market makers, who are willing to take on market-risk with a known, hedgeable portfolio of stocks.

I am fundamentally opposed to the 'Self-Indexing Fund' idea because it becomes all too convenient to hide active risk behind the mask of a "Silent Index". Gastineau alludes to the outperformance a Silent Index can offer, yet recent research by both Merrill Lynch and Goldman Sachs concludes that the S&P 500 index effects are diminishing, even as (or perhaps because) speculative capital targeting of major index changes has surged. I would argue that any divergent return is the result of active risk, and that the alpha could be either positive or negative. This ex-ante outperformance promise rings similar to the vibrant claims we heard in late-2000/early 2001, when the sell-side and competing index providers claimed that Provisional EAFE would defintitely outperform EAFE by 'at least' 100-200 bp, when in fact it underperformed in the first phase (May 31 2001 through November 30 2001) by 6 basis points. In fact, BGI calculated that when incidental country and sector bets (which provided a 35 bp gain) are stripped out, the net stock add/delete impact (i.e the pure "index effect") was a negative 41 bps.

Finally, the problem this solution attempts to address has not only diminished significantly as mentioned above, but is generally confined to the most popular indexes such as the S&P 500 and the Russell 2000. Many investors are actually moving to broader index strategies. These indexes are not affected by the sort of turnover that prompts the buying and selling that causes the index effect.

Peer-Based or Average Manager Indexes

The peer-based index proposes to use the determination of active mutual fund managers as a way of efficiently determining what asset classes are, and what benchmark and investable indexes should contain. This logic would deem that the appropriate benchmark should simply measure what active managers are holding in their portfolios, i.e. a variant of peer universes. While appealing in a zen-like way (it is what it is) this kind approach would risk mimicking active managers' tendency toward herd mentality, and insufficiently capture the true universe of the asset class.

The problem with this strategy is that ostensibly, the objective of both the theory and practice of index investing is to accurately reflect the investable opportunity set. Active managers tend to drift toward the latest fashion, and do not stay put in their pre-defined asset classes. Another practical problem is related to the challenge of "pre-defining" a size or style benchmark based on what managers actually hold, when the disclosure of these holdings at best has a significant time delay, and at worst, is opaque. As noted in above, transparency is vital for an efficient benchmark, yet peer-based benchmarks are by definition, ex-post. The alternative to this approach, of course, is a the true meritocracy of manager competition against an accurate investable-universe index.  

Total Market Benchmarks/Allocation and "Pure Passive" Portfolio Management Approaches

As an overall Investment Strategy, the concept of bypassing combinations of sub-asset class benchmarks (e.g. S&P 500 and Russell 2000) and replacing them with broad market indexes is quite sound, and this trend is strongly underway for both U.S. equity and internationalequity investment. The Russell 3000, Wilshire 5000, S&P 1500, DJTM, and MSCI ACWI ex-US and FTSE All-World have all made significant strides in attracting attention and assets recently. The low turnover, low costs, and high tax- efficiency associated with these indexes, together with their broad diversification, make them appealing for many investors.

The extension of broad market theory to index porfolio management has evolved so that certain index fund managers are promoting a "pure passive" style of indexation, essentially offering a "relaxed" approach to index changes, to the end of lowering turnover, potentially minimizing the "index effect" and ideally maximizing portfolio wealth. This concept makes good sense in theory, but it is still far from clear that investors will accept significantly higher tracking error (especially negative tracking) when they retain "brand name" indexes as their policy benchmarks. This potential outcome is similar to one of my concerns about Silent Indexes and the funds that may be based on them. Namely, in addition to potential cost/performance benefit from trading away from major index changes, higher tracking error will inevitably result from unintended (and therefore uncompensated) size/sector/style bets.This could result in underperformance well beyond the potential savings from avoiding specific index change events. Ultimately, as stated at the outset of this essay, competition between both benchmark methodologies and approaches to indexation is healthy, and I'm fully confident that the marketplace will decide the merits of the various products.

VI. Choosing the "Good" Index That is "Perfect" For One's Specific Needs

Now that I have elucidated a broad array of uses, attributes and inevitable tradeoffs of indexes and addressed the "silver-bullet " claims of a single perfect index approach, how do you choose your own perfect indexes to meet your specific needs?

The criteria generally accepted for choosing a broad-capitalization index depends on whether it is to be used as a benchmark for active management, as a portfolio (index fund), or as a proxy for an asset class in asset allocation. However, as noted earlier in the article, the best index is one that can be used for all three purposes simultaneously, so that one does not have to keep switching between indexes depending on the purpose one wants to use it for at a particular time. Obviously, this involves compromises, and thus it would be hard to expect such a multi-purpose index to be the "perfect" index for each of the individual objectives. A good metaphor would be a Swiss Army knife - it has very high utility for an array of tasks, but few of its tools are the absolute best for each specific use. Thus, most households will have a top-notch screwdriver, scissors, corkscrew, etc, in addition to the all-purpose knife. The broad-market index contains many of the tools that can serve your various investment needs, but you may also need a specialized - or even custom -- index to provide you with the efficacy of a real screwdriver for certain tasks. .

The alignment of overall investment policy with major benchmarks (strategic/tactical/implementation) provides enormous utility to investors/asset owners. Ensuring that your indexes are properly aligned provides significant value, and is in many ways the second most important decision after asset allocation. Few people in the financial industry care more about benchmark methodology than I do, and no firm cares more and does more about these issues than BGI. However, obsession with 'the perfect benchmark' is a misplaced use of industry resources, which could better be applied to continuous improvement of existing benchmarks. The industry would do better to focus on the development of custom solutions for specific client needs, and on refining the art and science of managing efficiently and consistently against the thousands of benchmark indexes available to investors.

Selecting an asset-class proxy

Some investors want the broadest possible index (such as the Wilshire 5000 or the Salomon Smith Barney (SSB) Broad Market Index (BMI) because they want the theoretically ideal "market portfolio" or because measures of aggregate wealth figure into their decision-making. Breadth, however, should not be the sole deciding factor. All of the genuine broad-cap indexes that the industry offers to investors have essentially the same long-term historical and expected returns, as well as similar risk and correlation characteristics. Given this, other criteria are more important. One should, for example, favor the index that has the longest and most accurate history, and other features (such as style and size sub-indexes, fundamental data, and industry and individual-company returns) that are important in one's approach to studying asset classes.

It is important, of course, that the selected index be representative of the asset class it is intended to represent. The S&P 500, for example, is not a broad-cap index and should not be used as a proxy for the full spectrum of U.S. stocks. For international equities, some of the same principles apply. While the SSB BMI covers 95% of the investable universe, the 80-90% coverage of FTSE, MSCI, Dow Jones Global, S&P Global and SSB Primary Market Index (PMI) is generally sufficient for an asset class proxy.

Selecting an active benchmark or index fund/ETF

When one is actually investing money, operational issues come to the forefront in selecting an index. The criteria for selecting an active benchmark and for selecting an index fund are closely related, because if one had no views on any stock, the active portfolio would presumably be identical to the index fund. The one difference is that active managers need more detail (including fundamental, industry, and company data) so they can evaluate bets made against the index and conduct performance attribution studies. For ETF products, the benchmark can be narrower, both in overall cap coverage, but also in country/sector/style segments.

Other operational issues

Investors should choose an index that is easy to use. Some indexes are better supported by the index provider than others; for example, some of the public Web sites providing return and constituent data are more complete, accurate, timely, and convenient than others. As noted earlier, the existence of clear, objective, and widely-disseminated rules for stock addition and deletion (and for other index-maintenance actions) makes it more practical to manage the fund to the index, or to use the index as a benchmark. Clearly, flexibility and responsiveness of the index provider is also essential for creating and maintaining custom benchmarks to meet specific investor needs.

Finally, as noted in the Seven Key Criteria section, all other factors being equal, a high degree of acceptance by the broad investment community makes an index significantly more useful and valuable. 

VII. Conclusion - - Near Perfect Choices in an Imperfect World

Ultimately, as stated in the Introduction, an open marketplace of index products (benchmarks and funds) with transparency of methodology and constituents, coupled with competition between both index calculators/vendors and index fund managers will result in the continual improvement of benchmarks and the products based upon them. This also ensures that benchmarks respond to the constant changes in the underlying markets, such as cross-border mergers, country graduations and the growth of new industries and sectors.

A diversity of index choices is essential, because each investor has unique needs, and these are best served by careful consultation with colleagues, managers and consultants. Furthermore, fund managers and asset owners should continuously make their views known to index vendors. The index providers should be open to this crucial input, as it leads to continuous index improvement. Finally, the continual development of "alternative" approaches to index portfolio management -enhanced indexing, "pure passive" strategies, alternative weighting approaches and even "silent indexes/ funds" -will ensure that the marketplace will continue to assess the attributes and tradeoffs of the varied approaches. In this robust environment of innovation and responsiveness, investors will actually be able to achieve their individually "perfect" benchmark and portfolio solution. Thus, through the efforts of "all of us", seemingly the "impossible dream" could be achievable.

Find your next ETF

Reset All