This is definitely an interesting set of issues to look at. Again, I think the closest ideas around what you're looking for are probably the fundamentally weighted indexes ... which show MUCH more stable sector weightings historically and largely avoided some of the worst excesses - like the huge tech bubble for example. Of course much of the data is backtested, but these are good, valid ideas in the right area.
You may already be familiar with the RAFI indexes or "indexes," but here's a link:
In particular, check out the sector charts on page 5 of the online version of the article...
While I'm not a fan of market-cap weighting, I'll quite gladly take it over approaches that address that matter but do so while taking on another form of extreme risk concentration.
If the issue is containing sector concentration, one need not resort to the RAFI sector series to get it done. Weighting within the sector index is a secondary matter to the question of weighting across sectors. But on the matter of internal weighting within the sector, the RAFI approach, by design, takes on extreme weightings in value characteristics that render it DOA for me as an alternative. And, if you wish to consider alternative weightings --- you can get a more reasonable/moderate portfolio-level profile of attributes just using the Rydex equal-weighted sector pieces.
Of the seven US sector based approaches, those that can make reasonable sense include Powershares Dynamic sector series, Rydex equal-weighted S&P sector series, iShares DJ sector series and Vanguard MSCI sector series. In other words, the only ones that I would categorically pitch into the dumpster are StateStreet's S&P 500, Powershares' RAFI and FirstTrust's Alphabits (yes, I meant to do that) --- purely on the basis of index attribute concentration issues specific to those series.
The info that I'm attaching (my messy work bench) needs quite a bit of cleaning up and updating so that all is expressed "apples-to-apples." I just haven't had the chance to "synchronize" it all and expand it in certain areas but I think that the somewhat messy picture regarding portfolio attributes, nevertheless, is still clear enough for some crude comparative work.
Over the last three years I've accumulated a fair amount of information on Powershares "Dynamic Portfolio" methodologies --- both in written form and through discussions with Powershares people. And, I think, that the screening and weighting methodologies employed do a reasonable job of fencing off risk - including in the sector series, while maintaining credible market cap distributions and valuation metrics. I do have a big, big problem, though, with a lack of publicly available information regarding their index methodologies and index data histories --- These are things that users simply and reasonably ought to demand (but there's no focal point for such a voice in the industry). I think that sponsors and providers ought to be expected by the investing world to provide quite a bit of information as a baseline diet (both on statistics and on methodologies). On the sponsors' and index providers' side there are copyright and trademark laws. And regulators ought to push fund sponsors and index providers to the wall on the data and methodology fronts while helping to aggressively defend intellectual property rights. Period. And for those who wouldn't touch a product for lack of understanding and lack of information regarding the methodologies and attribute dynamics --- I think that such a view on life makes a lot of sense.
I don't have any problem with back-tested data, so long as it is put to an appropriate use --- such as reviewing variability in index characteristics over time. And by that I mean looking beyond the total return stats and into matters such as sector and industry variation, capitalization variation, P/E, P/B, P/S, RoE, etc., etc. All of that stuff is "there" but fund sponsors and index providers, in general, do very, very little about making it available so that the dynamic characteristics of meaningful attributes can be thoroughly reviewed and vetted. Too many people, of course, focus first (and often exclusively) on total return historicals. And if that's all that is considered then one can paint just about whatever total return picture is desired --- and that sort of "artwork" is a serious problem for everyone.
Think about this ... Of all the attributes that can be isolated in a tradable, investable form - if the single most important driver of broad index performance (total market, style and other such approaches) among these is sector composition then sector composition makes sense as a practical, executable means to allocate and fence off risk. Whether one chooses to market-cap weight, equal-weight or take another alternative approach to weighting (e.g. Powershares dynamic sectors or RAFI sectors) within a given sector is a question of the second or third order --- an important one, but not the first one.
Regarding the article, there's some useful observations but I also think that there were some incredibly important angles completely, completely missed --- There was no attention or review given to equal-weighting or to alternatives focused on sector constraints as the first order of business. Regarding the former, if the broad indexes used for comparison relative to RAFI were able to be "objectively" scaled and then equal-weighted how would RAFI's brain surgery stack up (on historical total returns, since that was the primary validating criterion used in the article, but also across a range of portfolio attributes)? As for the latter, the article at least hints that important byproducts of the RAFI methodology are, effectively, sector constraints that impact performance. Voila / Eureka !!!!!!!!!!!!!! ... see below (AND the sector charts in the article) ...
"One intuitive way to understand why fundamental indexes outperform cap indexes is to examine the volatility of the cap index's sector exposure vs.. the fundamental index's sector exposure. We show the sector exposures of the U.S. Fundamental Index 1000 vs.. a cap-weighted peer benchmark in Figures 7 A-B.
Observe that the cap-weighted index's exposure to the tech sector ballooned to almost 25 percent of the entire portfolio at the height of the bubble ( from an eight percent exposure in 1995). Compare that to the fundamental index's exposure of 10 percent in 2000 (from six percent in 1995). Cap indexes, like the Russell 1000 and S&P 500, were drawn into the tech bubble as the tech names rapidly appreciated in price and market cap. As P/Es increased into the stratosphere, more and more index money was forced into Internet and telecom companies, which in turn pushed the valuation level on those companies higher, fueling even higher P/Es. Fundamental indexes do not experience this problem. Under the fundamental indexing methodology, unless companies grow their cash flows, dividends, sales and book value faster than the rest of the economy, they do not receive additional allocations. In a fundamental index, stocks that were popular and trading at a premium in 1999, like Cisco, Worldcom, AOL and other tech names, would not receive overweights at the expense of the utilities, financials and energies, which produced larger and more robust cash flows."
So, if sector composition is a primary driving issue, why not start the carving there? Clearly a key outcome of the RAFI screenings applied is a sector profile that moderates the extremes that are part and parcel of the broad indices. Take a look at the S&P equal-weighted ETF vs SPY (and, if you can, take a look at the composition of each from June 2007). It's true there too (as of 7/31 - using Morningstar data, Energy, for example, is just more than half in RSP relative to SPY).
Why make things so blasted complicated right out of the chute? This really is neither magic nor brain surgery. But it (RAFI non-sector constructs) is, rather matter-of-factly, a sector constraining operation in its outcome. I'd just throw in, though, that there are other characteristics of the output (such as extreme value weighting, even in the RAFI sector pieces) which ought to be weighed as well.
Take the time to walk through the same analyses referenced in the article but while wearing a set of sector glasses and I think you're likely to find that simply using sector constraints as the starting point provides a more clear, less "subjective" approach to alternative weighting. Isn't it cleaner and more practical to consider alternative weighting approaches by starting with constraint of weights across sectors and, only after that, moving on to how securities are weighted (market cap or otherwise) or screened within a given sector?
Shall we dive further into this with explicit consideration to the international front?
Perhaps the most compeling allocation-related research piece that I've come across in years is the one to which Matt provided a link a couple of months ago (Country, Sector or Style: What Matters Most When Constructing Global Equity Portfolios? An Empirical Investigation from 1990-2001). It also helps, in terms of the perspective taken, to strip away much of the bark on alternative weighting and "fundamental" indexing strategies.