The Wilshire 5000 Total Market Index

July 01, 2003


The Wilshire 5000 Total Market Index, commonly known as the Wilshire 5000, represents the entire U.S. equity market and is comprised of all publicly-traded stocks. Introduced in 1974, the Wilshire 5000 was constructed to serve four main constituencies: economists, practitioners, academicians, and legislators. Many of these professionals have embraced the Wilshire 5000 as a measure of wealth for the U.S. economy and also find it invaluable for any meaningful research of the stock market as a whole.

Because of its broad diversification and low turnover, the Wilshire 5000 has become increasingly popular with the investment management community. All of the major index fund man-agars have created successful funds that track the Wilshire 5000 within 10 to 30 basis points annually. This article will describe the portfolio management techniques used to successfully man-age a portfolio indexed to the Wilshire 5000 as well as argue the case for using the index as a benchmark for the 'market portfolio' as described by the Capital Asset Pricing Model. 1

Market Comparisons

Since the S&P 500 Index was (and mostly still is) the main barometer of U.S. equity investment performance from an institutional perspective, in the early 1970's Wilshire perceived an interest in looking at the U.S. equity market in its entirety, given that nearly 50% of the total market capitalization was being ignored at the time. Today, the S&P 500 represents approximately 80% of the Wilshire 5000 Index, a nearly 30% increase in the bias of the U.S. stock market to large-capitalization securities.

At first glance, this increase would seem to indicate that the S&P 500 Index is broad enough to capture the behavior of the market. Upon closer analysis, however, when the two indexes are compared over time, significant differences in the expo-sure to the overall market can be observed that leave unsuspecting investors open to additional market risk.

Figure 1 depicts rolling one-year correlation coefficients produced by regressing the total returns of the S&P 500 Index against the total returns of the Wilshire 5000 over 28 years. A correlation coefficient of 1.0 indicates that the two indexes have perfectly matched return behavior. Any value less than 1.0 indicates a deviation in that behavior. Figure 1 demonstrates that while there are periods where the indexes are perfectly correlated, there are several instances where correlations drop significantly.

Institutions attempting to capture the market return would miss out on the performance potential and, conversely, incur unintended risk in their overall allocation to the U.S. market by not being benchmarked to an index that has full exposure to all market sectors.

Most fund sponsors make allocations to the U.S. equity market that include style exposure and capitalization exposure (large, mid- and small). If the overall benchmark for the total fund does not capture the actual investment opportunity in the market, then unintended bets in the overall portfolio expose the fund to greater risk.

Figure 2 further illustrates the risk not captured by a fund whose overall U.S. equity benchmark does not include the full investment opportunity set. Figure 2 depicts rolling 1-year standard deviations of returns for the Wilshire 5000 and the S&P 500 Index.


Managing a Wilshire 5000 portfolio can seem daunting given the number of stocks that comprise the index. Portfolio managers have expressed concerns that the index is not liquid in the bottom tiers of capitalization and therefore other broad market indexes are more suitable. It has been demonstrated in earlier paragraphs that to view the S&P 500 as a broad market index excludes a significant amount of information about the behavior of the U.S. stock market. Essentially, institutional managers are incurring additional risk by creating an artificial bias to large-capitalization stocks and taking unintended sector bets relative to the market as a whole.

Figure 3 shows a snapshot of market liquidity over the past seven year ends starting in 1995. For each period, the stocks in the Wilshire 5000 are grouped by deciles of market capitalization. Each decile group depicts the capitalization-weighted aver-age of the group's constituent stocks' volume for that month-end. The average Wilshire 5000 portfolio contains stocks in the first five deciles. Only recently, with the reduction in the number of stocks in the total index, has it been necessary to trade in the seventh decile.

Figure 3 demonstrates that there is sufficient liquidity in this market cycle down to the fifth decile. In certain market cycles, trading volume is relatively constant through all tiers and even in thinly traded periods like 2001 and 2002, liquidity in the fifth tier is sufficient for a Wilshire 5000 fund. This point is further illustrated by the annual tracking error of 10 to 30 basis points achieved by the average Wilshire 5000 index fund.


The Wilshire 5000 has unique characteristics that make managing a fund against it relatively easy. The index has no artificial cutoffs in terms of security count. All of the other broad market indexes, such as the Russell 2000 and the S&P 1500 end at some pre-determined number of stocks that directly impact performance, volatility, and turnover.

Performance is impacted by these artificial cutoffs because an index will frequently miss out on gains from the emerging growth of a stock in the years following its initial public offering (IPO). It is very rare that an IPO's initial capitalization is large enough for immediate addition to other equity indexes.

Conversely, all broad market indexes, even those that limit the number of securities, will participate in losses that occur as a large capitalization stock falls from favor. While a decline in a stock's value is also experienced by the Wilshire 5000, the negative impact on performance is spread throughout the capitalization tiers.

Another impact from having artificial cutoffs is increased turnover and volatility. In active markets, stocks can frequently cross these cutoff thresholds and create unnecessary turnover and return volatility in the index. The Wilshire 5000 Index has very low turnover, as it is not necessary to sell any stock out of the index. If a stock leaves the index, it is either because it has stopped trading over two consecutive month-ends or it has been delisted. Turnover for the Wilshire 5000 was 3.93% for 2002 and the historical 5-year average turnover is 7.55%. Corporate actions are a headache for any index manager, and probably more so for a Wilshire 5000 manager. The greatest impact is still felt in the top two tiers of market capitalization; however, careful attention should also be paid to the lower tiers. Since the index includes every listed stock in the U.S. market, changes should occur naturally in the Wilshire 5000 port-folio without necessitating additional trading. It is important to review Wilshire's announcements to insure that the timing and description of the announcement are aligned with the manager's expectations.

Typical Fund Construction Methods

Wilshire Asset Management introduced the first index fund managed against the Wilshire 5000 in 1983. Since then, other index pioneers like Vanguard, Wells Fargo (now Barclays Global Investors), and State Street Global Advisors have also launched Wilshire 5000 funds. There are three portfolio construction strategies typically used to manage an index fund: linear optimization/stratified sampling, quadratic optimization, and full replication. Each method has advantages and disadvantages, with the ultimate goals of low tracking error and low turnover to deliver the index return at the lowest possible cost.

Full replication, where every stock in the portfolio is held at its exact weight in the index, is not a feasible strategy for man-aging a Wilshire 5000 portfolio of institutional size. Low liquidity for the smaller capitalization stocks in the index make them very expensive if not impossible to own. Further, it is not necessary to invest in the lower tiers of capitalization to success-fully deliver the index return, as long as the portfolio has the correct exposure to the characteristics of the index that drive performance.

Quadratic optimization is the tool most often used by active managers for portfolio construction. While index fund managers do employ this method, a quadratic optimizer is designed to buy the entire index or benchmark because by definition, that is the portfolio with the lowest tracking error. Since full replication is not practical or necessary when managing a Wilshire 5000 portfolio, constraints must be applied within the optimization in order to achieve a more meaningful solution. Wilshire also offers clients a hybrid solution that combines the two methods and can result in a portfolio that provides low tracking error but contains fewer stocks.

These methods will be discussed in greater detail in the following paragraphs with the strengths and weaknesses being judged by a single criterion: tracking error. Tracking error emerges from three main sources: transaction costs, compositional differences between the portfolio and the index, and pricing differences on trades.

Stratified Sampling

Stratified sampling, developed by Wilshire Associates in 1976, is the most widely used approach for managing a Wilshire 5000 portfolio. It is also the most straightforward and easiest to implement. The typical fund manager will manage a portfolio containing between 2,900 and 3,500 stocks. The sampling approach divides the index into cells. Using Wilshire's methodology, these cells usually represent industry sectors and market capitalization rankings. Wilshire divides each industry sector into ten deciles of market capitalization and then holds stocks in each cell at their appropriate weight.

This strategy seeks to minimize tracking error by 'stratifying' the market capitalization of the index across the optimal number of cells. The number of cells necessary to define an index is directly proportional to the number of stocks that comprise that index. The Wilshire 5000 requires, at minimum, an industry classification system narrow enough to capture the homogeneous behavior of each group's representative stocks but not so narrowly defined that there are not enough stocks represented in each group.

The recent introduction of four-tier industry classification systems such as the Dow Jones Global Classifications, offers the flexibility for portfolio managers to experiment between the second- and third-tier industry and sub-industry categories for more granular stratification.

The Wilshire sampling methodology uses 'posits,' or position size, as defined by a stock's percentage of total market capitalization to decompose the Wilshire 5000. A position size cut-off of around .007% is typical for a Wilshire 5000 fund. Stocks are ranked in the index by percent of market capitalization and at a ranking of .007%, the index is divided into two parts. Stocks with a ranking of .007% or higher are defined as 'core' securities and are held at their exact weight in the index. Stocks ranked below .007% are defined as 'non-core' securities. Non-core stocks are used to fill in the industry weighting gaps left by the core holdings. Non-core stocks are purchased in equal-weighted increments at a posit of .007% until each industry cell weight matches that of the index.

Figure 4 provides a snapshot of the characteristics of a Wilshire 5000 portfolio created using the stratified sampling approach. The Portfolio and Target columns indicate the weight of each industry for the portfolio and the index. The (P)-(T) column represents the difference in those weights. The Core, Noncore and Total column indicate the number of stocks represented for the portfolio in each industry. Select, Weight and Total represent a decomposition of the difference in return for the portfolio and the benchmark at the group level. Select refers to stock selection and measures the amount of excess return that can be attributed to mismatches in stocks held in the portfolio versus the index. Weight refers to return differences that are caused by mismatches in the weighting of each industry relative the index.

This Wilshire 5000 fund's industry weights are very closely aligned but not exact. The variance section of the report indicates that these slight mismatches in exposure have very little impact on the return variance of the fund versus the benchmark at the industry level over the one-month period of analysis.

Quadratic Optimization

Whereas stratified sampling seeks to match the cell weights in the portfolio with those of the benchmark, quadratic optimization produces a portfolio that minimizes or maximizes the value of some 'objective' function, in this case minimize tracking error, subject to a set of weight constraints. The objective function or one of the constraints has a term that is quadratic. Quadratic optimization can also be used to find a portfolio that is tilted toward a specific characteristic (i.e., yield) subject to a constraint on tracking error variance. Typically, the foundation of the optimizer is a risk model that is described by a variance/covariance matrix that contains factor variances and co-variances that are estimated from factor returns or stipulated a priori. In Wilshire's case, co-variances are estimated from factor returns that are derived from cross-sectional regressions of individual security returns on attributes (such as industry classifications and company balance sheet fundamentals).

Because tracking error variance depends on the exposure of the portfolio to the same kinds of factors that are frequently used to classify securities for stratified sampling, quadratic optimization will tend to find portfolios that are closely matched to the benchmark in the very same dimensions that stratified sampling does. The main advantage is that when there are two or more portfolios that match the benchmark in their exposures to the risk factors, quadratic optimization will select stocks in combinations that will minimize the predicted tracking error variance. The disadvantage of quadratic optimization is that this choice depends crucially on estimates of the factor return co-variances, which are subject to estimation error.

A common strategy for creating a Wilshire 5000 fund is to break it into two components: an S&P 500 fund (first component) and a fund based on the Wilshire 4500 Completion Index (second component). Many of the major index fund providers manage funds based on sever-al U.S. market indexes, so it is quite common for them to offer both fund types as separate products. A full replication strategy is then used for the S&P 500 fund since it contains highly liquid stocks, and quadratic optimization is employed to create the Wilshire 4500 fund. This practice can be used as a single strategy or in a 'fund-of-funds' approach where the funds are actually managed separately and then aggregated. A secondary rebalance at the total fund level is necessary to insure that the funds are combined properly so that the component weights match the overall Wilshire 5000 portfolio according to the market value of the total fund.

As of December 31, 2002, the Wilshire 4500 contained 5,168 stocks. Since the quadratic optimizer may try to buy every security in the benchmark if the objective is set simply to minimize tracking error variance, some additional constraints must be applied to achieve a more manageable solution. One way to accomplish this is to decide on a risk tolerance level or a minimum acceptable tracking error variance and use the optimizer to find a portfolio that has a minimum or maximum value of some other characteristic, such as yield, but is constrained to have no more than the specified level of risk. Additionally, by employing liquidity screens on trading volume information for stocks in the candidate universe, the fund manager can help the optimizer avoid selecting illiquid stocks in the optimal portfolio. Most U.S. risk models are derived from a universe of the top 1,500 to 2,500 stocks ranked by market capitalization.3

This universe is restricted due to missing data or poor data quality in the lower tiers of market capitalization in the Wilshire 5000. It is common for most variance/covariance matrices to be constructed with at least 60 months of historical data, and if more factors are included in the model, more data points are required to keep the matrix square. Smaller capitalization stocks in the lower tiers typically have not been trading (or incorporated) long enough to build the required history necessary to be included. Therefore, when creating a fund (using a quadratic optimizer) for the Wilshire 4500, constraints as well as some active decisions must be used to achieve the optimal portfolio. The return behavior of stocks in the lower capitalization tiers is mostly derived from specific risk, or risk not explained by factors in the risk model. The optimizer will buy these securities in great numbers if unconstrained, to insure that the total expo-sure replicates the factor exposure of the index. The impact of these securities may be insignificant but some return surprises, either negative or positive, can occur if the positions are not managed carefully. Constraints must be set for liquidity, industry weight ranges, and possibly for minimum security weights in order to limit the number and impact in terms of volatility and trading effects of the stocks the optimizer suggests. Figure 5 below displays the output from an initial quadratic optimization on the Wilshire 4500 starting from a cash portfolio of $100,000,000. Definitions for the technical terms are avail- able in Appendix 2.

Figure 5 displays the results from an optimization to create a Wilshire 4500 portfolio. The annualized expected tracking error for the portfolio is +/- 49 basis points and the Coefficient of Correlation is exactly one, which indicates that if the market behaves in a similar manner to the historical data represented by the variance/covariance matrix, this port-folio should track the actual Wilshire 4500 Index within 50 basis points for the next 12 months.

Hybrid Processes

Construction methodologies that employ variations of the three methodologies are also popular for managing a Wilshire 5000 fund. Full replication in combination with quadratic optimization has been previously mentioned. Another hybrid method involves the use of both stratified sampling and quadratic optimization. Stratified sampling is used to obtain the core stocks in the portfolio; then quadratic optimization is used for the non-core universe. Typically, the non-cores are more actively managed using this hybrid methodology. The portfolio manager will calculate an alpha (based on some proprietary metric) or use the expected returns for each stock in the objective function of the optimizer. The optimizer can be directed to maximize the objective function (or, in this case, alpha) while minimizing tracking error. While the optimizer iterates to find the optimal portfolio, it will create various portfolios at each pass that can fit the optimal criteria. If there is a solution with a higher alpha and equivalent tracking error, then it will choose the higher alpha portfolio.

Index fund managers may employ this technique more aggressively for enhanced index strategies. Traditional index fund managers use this technique to create a positive performance cushion to insulate them from any gaps in their portfolio's characteristics versus the index that might lead to negative tracking error. While the ultimate goal is zero tracking error minus transaction costs, since full replication is not a solution for a Wilshire 5000 portfolio, there will be variance in return versus the index in any given month. This variance should be mean reverting if the index characteristics are efficiently managed within the fund. Performance surprises can occur, however, and a positive return cushion can help mitigate any trend in the return bias of the fund.

Since the Wilshire 5000 has no artificial cutoffs, it is a very fluid index. Stocks are only removed from the index if they stop trading for two consecutive months, which allows a security to migrate freely, in any direction through the index as its fortunes increase or diminish. The effect of this freedom, from a management standpoint, is that the only difficult part (and only mildly so) of managing a Wilshire 5000 index fund is during its initial creation. The degree of difficulty depends on the method the portfolio manager selects to construct it as well as the amount of assets on hand to start the fund. After the fund has been constructed, maintaining it is a fairly simple process. Rebalancing Strategies

Once the initial fund is created, the rebalance frequency should be very low if the characteristics of the fund are closely aligned with those of the index. Wilshire 5000 managers rebalance because of contributions and withdrawals to the fund, cash buildup due to dividend distributions, and IPOs entering the index.

Regardless of the portfolio construction method used, Wilshire 5000 managers typically run their analysis tools frequently to check the status of the fund. Managers who use Wilshire's rebalancing tool can check reports designed to highlight differences in the fund (such as the report in Figure 4). If the report indicates a difference in either the weighting scheme or the portfolio's return versus the benchmark, rebalancing the portfolio may be necessary. These reports also seek to pinpoint the exact location of the mismatch to assist managers with the rebalancing decision. Ideally, the index fund manager avoids rebalancing unless absolutely necessary. Wilshire's stratified sampling methodology is designed to reduce transaction costs by implementing bands around key weighting criteria. For example, core positions have bands of 1 posit (equal to 1% of total portfolio market capitalization) above and one-half posit below the actual weight of a stock. A stock may float within this band before a trade recommendation is generated. If a non-core stock's weight is between 0 and 2 posits, it is considered to be with-in an acceptable range. This methodology allows the stock to participate in a reasonable amount of upside growth while reducing negative return exposure. These boundaries may be relaxed if less trading is desired.

Quadratic optimization tools require more attention during the rebalance process, as the manager may have to use different constraints by trial and error to find a satisfactory trade-off between producing the optimal portfolio and limiting trades. Constraints that 'keep' certain securities in the portfolio, or that require a certain number of stocks, may be necessary in conjunction with the minimum weight constraints employed during the initial optimization. Hybrid strategies may have a combination of the issues outlined above or require even more attention if an alpha strategy is being used.

In addition to monitoring the rebalance frequency, managers pay attention to how they trade securities. In general, index fund managers emphasize efficient, low-cost trading programs. Wilshire 5000 fund managers are particularly careful, especially when trading in the lower capitalization tiers.

When possible, futures contracts will be used to 'equitize' any cash in the portfolio or for managing transition portfolios in order to maintain market exposure. Index managers that are large enough in terms of breadth of product offerings and assets under management will manage trading by swapping stocks from one portfolio to another in order to match contributions in one fund with withdrawals from another. Other managers will buy larger positions of more liquid non-core stocks to protect industry weights to give them more time to cover less liquid, non-core trades. An important component to achieving a lower cost transaction is to have as little market impact as possible. The last thing an index manager wants is for the market to move during the middle of a trade.

An important side benefit to managing a Wilshire 5000 portfolio is tax efficiency. As previously discussed, the Wilshire 5000 is a very fluid index since there are no artificial cutoffs in its design. The implications are that there should be very little, if any selling within the portfolio, which means no capital gains distributions for mutual funds, only dividend distributions.


The broad diversification, low turnover, low cost and excel-lent tax efficiency of total stock market funds have become increasingly popular with both institutional and retail investors. As the only index representing the entire stock market, and by definition containing the broadest possible diversification and lowest turnover, the Wilshire 5000 is an ideal market benchmark and fund portfolio. The preceding pages have demonstrated the only requirements to run a Wilshire 5000 portfolio are assets and a few in-house tools to assist with fund construction and rebalancing. For all of these reasons, we anticipate that the Wilshire 5000 Index fund will continue to thrive as a core strategy for both institutional and retail equity portfolios.


Special thanks to Robert C. Kuberek, Steven J. Foresti, John T. Winslow, Lawrence Mano and Karl Schneider for their contributions to this article.

This article was reviewed by Sanjay Arya, Heather Bell and Peter Vann.



1. Sharpe, William, 'Capital Asset Prices: A Theory of Market Equilibrium,' Journal of Finance, September 1964; Lintner, John, 'The Valuation of Risk Assets and the Selection of Risky

Investment in Stock Portfolios and Capital Budgets, ' Review of Economics and Statistics, February 1965; Mossin, Jan, 'Equilibrium in a Capital Asset Market', Econometrica, October 1966.

2. Wilshire's clients that prefer this method use our U.S. risk model in conjunction with the quadratic optimizer found in the Wilshire Atlas. Factors in Wilshire's U.S. Risk Model include

Wilshire industry classifications, three style factors, and three momentum factors.

3. Wilshire's Risk Model Universe contains 2,500 stocks for the U.S. market.

Special thanks to Robert C. Kuberek, Steven J. Foresti, John T. Winslow, Lawrence Mano and Karl Schneider for their contributions to this article.

Find your next ETF

Reset All