Research Affiliates: Improving On Traditional Commodity Indices

Traditional long-only commodity indices tend to impose negative roll returns on investors due to construction rigidity.

Reviewed by: Jonathan Treussard
Edited by: Jonathan Treussard

Commodity markets have served a critical function of risk mitigation and risk transfer for thousands of years. In fact, derivatives trading in commodities was formalized in one of the first recorded legal codes, the Mesopotamian Code of Hammurabi, around 1750 BC (Oldani, 2008, p. 2–3). Today, as in the past, commodities play a key role in controlling risk for both commercial businesses and investors. Being real assets, commodities provide inflation protection and valuable diversification benefits to traditional core stock and bond portfolios. Yet adoption by both individual and institutional investors remains relatively limited for a variety of reasons, which include their “exotic” standing among asset classes, essentially, a fear of the unfamiliar; the perceived complexity of investing in futures contracts; and the performance headwinds of traditional, long-only commodity indices, such as the first-in-its-class S&P GSCI.

While having delivered substantial protection against rising inflation, traditional commodity indices have produced disappointing returns in benign inflation environments, encouraging many investors to think twice about a strategic allocation to commodities. That said, a long-only commodities portfolio can be a compelling long-term investment, assuming it is designed in a way that reverses the curse of unnecessarily poor returns by taking advantage of robust sources of excess return. In this article we review the case for commodities and the design advantages of the Dow Jones RAFI™ Commodity Index.

The Positive Attributes Of Commodities

Most individuals and businesses hold insurance policies to mitigate the high costs associated with unexpected events, such as car accidents, home fires, lawsuits, and medical diagnoses. Some types of insurance are mandated by law (e.g., car insurance), but the majority of policies are voluntarily purchased. Insurance premiums are a necessary expense for often (after the fact) unneeded insurance, but a cost well worth bearing if the event insured against unfolds. The same largely applies to commodities: the asset class acts as a diversifier because of its low correlation with other asset classes as well as acts as insurance against rising inflation due to its positive correlation with inflation (Greer, 2006). Looking back almost a half-century from January 1976 to June 2016, the monthly correlation of the S&P GSCI with the S&P 500 Index was 0.17, and the monthly correlation of the S&P GSCI with the US Consumer Price Index (CPI) was 0.46.

Commodities as diversifier. Adding a 5% commodities allocation to a traditional 60% US core stock/40% US core bond portfolio (i.e., 55% stocks/40% bonds/5% commodities) can markedly reduce volatility. Over the period January 1979 to June 2016, the volatility reduction, or the simple difference between the rolling three-year volatility of a 60/40 portfolio and the 55/40/5 portfolio, lowered overall portfolio volatility by an average of 53 basis points (bps) a year. In the late 1990s, the diversification benefit rose to over 100 bps. The sole exception of commodities not adding any benefit was during the global financial crisis when every asset class moved downward in unison.


Research Affiliates: Improving On Traditional Commodity Indices

For a larger view, please click on the image above.



Commodities as inflation protection. We can assess commodities’ inflation protection ability by looking at performance (i.e., insurance payout) during periods of high inflation, which we define as 3% or greater, occurring in roughly one-third (62 of 199) of the trailing 12-month periods during these years. The average excess return (net of the risk-free rate) of a US 60/40 portfolio is a relatively low 1.4% compared to 24.2% for the S&P GSCI over the period January 1999–June 2016. But when inflation was low, defined here as less than 3%, the first-generation commodity index generated an average return of −7.3%.


Research Affiliates: Improving On Traditional Commodity Indices

For a larger view, please click on the image above.


Where Traditional Commodity Indices Fall Short

Given that traditional commodity indices have generally delivered on their diversification and inflation-hedging benefits over the last 40 years, what explains the muted adoption by investors? In a word: performance. If we plot the growth of $100 from January 1976 to June 2016 invested in the S&P GSCI, US 60/40 portfolio, US T-bills, and US CPI, the commodity index’s ending value of $595 underperforms both the 60/40 portfolio ($5,015) and T-bills ($691), and beats inflation ($430) by only $165 as of the end of June 2016. The annualized performance of the S&P GSCI was 4.5% compared to 10.2% for the 60/40 portfolio and 4.9% for T-bills. The annualized inflation rate over the period was 3.7%. Simply put, investors in the S&P GSCI earned a return lower than T-bills despite being subjected to annualized volatility of nearly 20%.


Research Affiliates: Improving On Traditional Commodity Indices

For a larger view, please click on the image above.


Traditional commodity indices suffer from two drawbacks that negatively impact their performance: 1) a lack of diversification and 2) exposures that do not adjust with market conditions.


Lack of diversification. The weighting methods of traditional indices solely based on measures such as world production values—the Bloomberg Commodity (BCOM) index being a notable exception—tend to create very large tilts toward energy markets. The sizeable energy concentration of traditional commodity indices can lead to significant risk of large downturns when macroeconomic events adversely impact the energy markets, such as in 2008 when the S&P GSCI fell 46.5%, and again in 2015 when it dropped 32.9%. Six separate commodity contracts are associated with energy, but with the single exception of natural gas, all are very highly correlated with crude oil, being derived directly from it.1 Therefore, by focusing so heavily on one highly correlated sector, investors are foregoing meaningful diversification benefits.


Research Affiliates: Improving On Traditional Commodity Indices

For a larger view, please click on the image above.


Ignoring market conditions. In the 1920s John Maynard Keynes developed the theory of normal backwardation. Keynes argued that commodity producers would theoretically be willing to pay a premium to buyers of commodity futures in order to hedge away the risk of falling commodity prices in the future. He argued that it would be normal for commodity futures prices “out on the curve” to be below the spot price, and named this condition backwardation.2

Whereas backwardation may seem a logical market outcome, commodities also exhibit a condition called contango, the opposite of backwardation, in which the spot price is higher than the futures price out on the curve. Nowadays, many commodities are frequently in contango due to storage costs and shifting relationships among suppliers, producers, and speculators, and this contango is usually steepest at the front of the curve. The traditional commodity index construction repeatedly buys the most liquid front-month futures contracts, which induces a high cost of rolling contracts in times of contango and generates significantly negative roll returns.


Improving Commodity Index Design

Commodity index design, and correspondingly long-term expected performance, can be improved in two important ways: weighting and contract selection.

Weighting. An overweight to commodities in backwardation (or in less-extreme contango) in order to capture a relatively high and attractive roll yield, and an overweight to commodities with higher recent performance in order to benefit from short-term persistence in commodity price movements (i.e., positive momentum), can have meaningfully positive impacts on portfolio performance.3,4 We compare the performance of four portfolios—high versus low roll yield and high versus low momentum—from January 1999 to June 2016.


Research Affiliates: Improving On Traditional Commodity Indices

For a larger view, please click on the image above.


The portfolios are constructed as follows: 1) the high roll-yield portfolio consists of the top one-third of commodities with the highest roll yields over the period, 2) the low roll-yield portfolio consists of the bottom one-third of commodities by roll yield, 3) the high-momentum portfolio consists of the top one-third of commodities by recent performance, and 4) the low-momentum portfolio consists of the worst performing one-third of commodities. All portfolios are rebalanced monthly. Commodities in each portfolio are equally weighted.

Going long the commodities with the highest roll yields and shorting those with the lowest roll yields over the January 1999–June 2016 period produced a 14.04% annualized return advantage. Furthermore, the best roll yields may at times include those in contango (just less-steep contango), and the worst may include lesser-backwardated curves. In the same vein, a 10.28% return difference was earned by going long the commodities with the highest momentum and shorting those with the lowest momentum.


Contract Selection. The second improvement in commodity index design is in contract selection, moving beyond the so-called front contract (the futures contract closest to maturity) to buy liquid contracts with better local roll-yield profiles relative to the rest of the futures curve.

In contrast, traditional commodity indices commit to the front contract only, despite the significant losses created by the approach when the front end of the curve is in strong contango.

Analyzing the period from January 1999 to June 2016, we find that selecting contracts with the best roll-yield profile has historically improved returns by 3.8% (4.0% – 0.2%) a year compared to the return of a contract selection method that buys the front contract only. Such an improved return is higher than we would reasonably expect going forward, but additional worthwhile benefits can be captured using this form of contract selection. Because the front-month contract is often the most volatile, removing the obligation of buying the front contract can also reduce the volatility of the commodity investment. For example, the annualized volatilities of the two contract selection methods over the study period are 25.0% and 29.0%, respectively, indicating a reduction in volatility for the best roll-yield-profile selection method. The best roll-yield approach also cut turnover by one-half, on average, from 69% to 34%, and also typically lowers transaction costs.


Research Affiliates: Improving On Traditional Commodity Indices

For a larger view, please click on the image above.



Putting It All Together: The Dow Jones RAFI Commodity Index

The Dow Jones RAFI Commodity Index (DJ RAFI Commodities), which went live in September 2014, uses the previously discussed insights to build better return potential for commodity index investors and for traditional 60/40 investors. DJ RAFI Commodities achieves the traditional inflation protection and diversification benefits of commodity indexing, while mitigating the impediments of more traditional indices.

The index is constructed with the following methodology:

  • Uses base weights of 24 commodities across three major sectors, determined annually, from the parent Dow Jones Commodity Index.5
  • Reweights the portfolio (i.e., modifies the base weights) monthly based on an equal blend of momentum and roll-yield signals.
  • Selects contracts monthly, after the monthly portfolio weights are determined (i.e., the index selects the contract with the highest implied roll yield from a set of liquid contracts up to 24 months out on the futures curve).6

DJ RAFI Commodities performed well when compared to the S&P GSCI and the BCOM over the period January 1999–June 2016. A $100 investment in DJ RAFI Commodities in January 1999 would have grown to $645 by the end of June 2016, producing an annualized return of 11%. Similar investments in BCOM and S&P GSCI over the same time period would have grown to $159 and $121, respectively. DJ RAFI Commodities added 10% a year in excess return relative to the S&P GSCI and 9% a year over BCOM.


Research Affiliates: Improving On Traditional Commodity Indices

For a larger view, please click on the image above.


Importantly, this enhanced index construction does not adversely impact the core benefits of investing in commodities. Repeating our earlier exercise of assessing performance in high- and low-inflation environments, when we add DJ RAFI Commodities to our analysis we find an improvement in performance persistent in both inflation environments. This characteristic resembles what could be viewed as an “inflation insurance” policy that offers generous damage coverage and a negative premium.


In the parlance of risk management, DJ RAFI Commodities could be classified as an “alpha-generating hedge.” The index exceeded the average excess return of the BCOM by 15.7% a year and by 5.4% a year compared to the S&P GSCI during periods of high inflation, as well as outperforming during periods of low inflation, generating an excess return of 3.1% compared to the returns of −7.3% and −3.4% of the S&P GSCI and BCOM, respectively.


Research Affiliates: Improving On Traditional Commodity Indices

For a larger view, please click on the image above.


Since the launch of DJ RAFI Commodities in September 2014, a period that has encompassed the very turbulent commodity markets of 2015, the index has outperformed both traditional indices. An initial investment of $100 in DJ RAFI Commodities would have dropped to $75, a loss of $25, by the end of June 2016. Over the same period, an equal investment in the S&P GSCI and BCOM would have generated losses of $50 and $70, respectively. When investors experience only a relatively smaller loss, it can, of course, feel like a consolation prize at best. But for investors with a strategic allocation to commodities, the fact that DJ RAFI Commodities was able to “add” since launch an average of 17% a year relative to the S&P GSCI, and 3% a year against BCOM, can be viewed as a testament to the RAFI methodology, and points to the potential for more tolerable performance even under difficult circumstances.


Research Affiliates: Improving On Traditional Commodity Indices

For a larger view, please click on the image above.



Commodities can serve as an effective portfolio diversifier and can provide inflation protection. Traditional long-only commodity indices suffer, however, from a lack of diversification, relying on a handful of return drivers, and tend to impose negative roll returns on investors because of construction rigidity regarding weights and contracts. The Dow Jones RAFI Commodity Index is built to seek better performance from commodity investing, preserving the positive attributes of being long commodities and delivering superior cost-effective performance through a thoughtful, transparent, and rules-based investment process.



  1. Karstanje, van der Wel, and van Dijk (2015) examine common factors in commodity futures curves over the last 20 years or so, finding that, on average, 62% of the variation in level at the short end of the curve can be explained by a combination of market-wide and sector factors and that 74% of the variation in slope and curvature is captured, on average, by common factors. In addition, it appears that the importance of common factors for levels has increased, possibly as a result of the so-called financialization of commodity markets since the 1990s.
  2. For additional background on the posited reasons for expecting a return premium from investing in commodity futures, see Gorton and Rouwenhorst (2006).
  3. To define roll yield, we compare the front-month contract to its comparable contract 12 months out the curve to determine whether each commodity is in backwardation or contango. Using a fixed 12-month distance between contracts gives a measure that is more homogeneous across different commodities, eliminates seasonality in prices, and significantly reduces the volatility of roll yields, and therefore portfolio turnover. To assess momentum, we compare the spot price today to the price 12 months ago.
  4. Commodity prices tend to exhibit short-term momentum (prices that have gone up or down over the recent past continue the trend in the near future) in response to shocks in supply or demand. For example, the 2015 plunge in oil prices was largely a result of oversupply from fracking by US shale producers. In the United States, oil production rose from a previous low of 5 million barrels a day (mb/d) in 2008 to over 9 mb/d in 2015. According to the International Energy Agency, the implied global oil market surplus is expected to narrow significantly to 0.2 mb/d in fourth quarter 2016. Because shale producers have sunk costs in wells already drilled and will seek to make good on their investments, the delay in returning to a supply/demand balance will take time, creating persistence in the price of oil.
  5. The three major sectors include energy, agriculture and livestock, and metals. Each sector takes one-third of the total index weight. The base weights for the commodities in each sector are determined by a liquidity measure proxy of the five-year moving average of dollar-volume traded.
  6. The eligible contract may have open interest of at least 5% of the open interest of the nearby liquid contract. Contract eligibility is subject to annual review and adjustment as necessary for capacity. The contract roll takes place over five days beginning on the first Dow Jones RAFI Commodity Index business day of each month (i.e., on each roll day 20% of the incumbent contract is replaced with 20% of the new contract.)


Gorton, Gary, and K. Geert Rouwenhorst. 2006. “Facts and Fantasies about Commodity Futures.” Financial Analysts Journal, vol. 62, no. 2 (March/April):47–68.

Greer, Robert J. 2006. “Commodity Indexes for Real Return,” Chapter 5 in The Handbook of Inflation Hedging Investments, edited by Robert J. Greer. New York, NY: McGraw-Hill.

Karstanje, Dennis, Michel van der Wel, and Dick van Dijk. 2015. “Common Factors in Commodity Futures Curves.” University of Pennsylvania working paper (February 15).

Oldani, Chiara. 2008. Governing Global Derivatives: Challenges and Risks. New York, NY: Taylor & Francis Group.



The material contained in this document is for general information purposes only. It is not intended as an offer or a solicitation for the purchase and/or sale of any security, derivative, commodity, or financial instrument, nor is it advice or a recommendation to enter into any transaction. Research results relate only to a hypothetical model of past performance (i.e., a simulation) and not to an asset management product. No allowance has been made for trading costs or management fees, which would reduce investment performance. Actual results may differ. Index returns represent back-tested performance based on rules used in the creation of the index, are not a guarantee of future performance, and are not indicative of any specific investment. Indexes are not managed investment products and cannot be invested in directly. This material is based on information that is considered to be reliable, but Research Affiliates® and its related entities (collectively "Research Affiliates") make this information available on an "as is" basis without a duty to update, make warranties, express or implied, regarding the accuracy of the information contained herein. Research Affiliates is not responsible for any errors or omissions or for results obtained from the use of this information. Nothing contained in this material is intended to constitute legal, tax, securities, financial or investment advice, nor an opinion regarding the appropriateness of any investment. The information contained in this material should not be acted upon without obtaining advice from a licensed professional. Research Affiliates, LLC, is an investment adviser registered under the Investment Advisors Act of 1940 with the U.S. Securities and Exchange Commission (SEC). Our registration as an investment adviser does not imply a certain level of skill or training.

Investors should be aware of the risks associated with data sources and quantitative processes used in our investment management process. Errors may exist in data acquired from third party vendors, the construction of model portfolios, and in coding related to the index and portfolio construction process. While Research Affiliates takes steps to identify data and process errors so as to minimize the potential impact of such errors on index and portfolio performance, we cannot guarantee that such errors will not occur.

Research Affiliates is the owner of the trademarks, service marks, patents and copyrights related to the Fundamental Index methodology. The trade names Fundamental Index®, RAFI®, the RAFI logo, and the Research Affiliates corporate name and logo among others are the exclusive intellectual property of Research Affiliates, LLC. Any use of these trade names and logos without the prior written permission of Research Affiliates, LLC is expressly prohibited. Research Affiliates, LLC reserves the right to take any and all necessary action to preserve all of its rights, title and interest in and to these terms and logos.

Various features of the Fundamental Index® methodology, including an accounting data-based non-capitalization data processing system and method for creating and weighting an index of securities, are protected by various patents, and patent-pending intellectual property of Research Affiliates, LLC. (See all applicable US Patents, Patent Publications, and Patent Pending intellectual property located at, which are fully incorporated herein.)

The views and opinions expressed are those of the author and not necessarily those of Research Affiliates, LLC. The opinions are subject to change without notice.

©2016 Research Affiliates, LLC. All rights reserved.

Jonathan Treussard, Ph.D. Senior Vice President, Product Management, Research Affiliates, LLC

Lillian Wu,  Senior Researcher, Research Affiliates, LLC


Jonathan Treussard, Ph.D., is senior vice president, product management, at Research Affiliates, LLC.