How To Evaluate Smart Beta ETFs

It’s hard to keep track of all the smart-beta ETFs hitting the market these days. But according to OppenheimerFunds’ David Mazza, there’s a repeatable process to follow in evaluating those funds.

In a conversation with Inside ETFs CEO Matt Hougan, Mazza, who is head of beta solutions investment marketing and ETF specialists at OppenheimerFunds, discusses his ETF due diligence policy, and examines whether today’s smart-beta strategies are getting “crowded.” Mazza is a featured speaker at the forthcoming Inside Smart Beta conference, taking place June 8-9 in New York City, where he’ll be digging into this topic at a deeper level.

Matt Hougan: All smart-beta products promise great returns. But one thing we’ve found is that the live results don’t always measure up to the backtests. Why is that? And should we worry about an “observation effect” ruining factor investing?

David Mazza: Investors really need to understand where smart beta fits in their portfolios and then use that realization to evaluate funds and answer those questions appropriately.

Smart beta has the potential to do three things for investors: It can offer the potential for increased returns, reduced risk or improved diversification. You have to understand your desired investment outcome before you evaluate a fund and its performance.

A low-volatility product may get beaten up in certain time periods; there may even be times when it has higher volatility than the market. But in the very long run—10 years, or more—it may offer a smoother ride. Viewed through that lens, I might accept some underperformance in the short term.;

The industry throws out the buzz term “outcome-oriented investing” quite a bit, but it’s actually quite relevant vis-a-vis smart beta. You have to understand the long-term return that you’re going for and hold these products accordingly.

Hougan: How do you make sure investors stick around for those long-term benefits?

Mazza: That’s the big question, and the answer is, you need truth in labeling and a real focus on education from ETF issuers, index providers and the financial industry at large.

ETFs are a disruptive technology, and smart beta is at the forefront of that disruption in terms of its impact on both the cap-weighted community and the active side. Fortunately, we are now moving beyond finger-pointing and are dug in on true due diligence around how to best use these products.

When I’m speaking with investors and looking to position either existing funds or new solutions in their portfolios, I’m going to focus most on why the underlying anomalies that drive factor returns exist, and highlight the outcomes that they deliver. That’s what will help them hold portfolios long enough to realize the benefits smart-beta strategies may deliver.

Hougan: Are any smart-beta strategies getting crowded? How would you tell if they were?
Mazza: Investors should be cognizant of the potential for crowding in particular factors, but I think we’re a long way away from the point where we’re seeing the underlying anomaly for any particular factor being arbitraged away.

If a factor is highly valued, I may need to dampen my expectations for future returns. But if the underlying driver of the factor still exists, the long-term outcome should be there.

Beyond that, there’s a lot of exciting work being done by academics and practitioners on how different factors perform in different macroeconomic environments. That’s definitely a space worth watching.

Hougan: How would we know if an anomaly was arbitraged away?

Mazza: I’m a big believer in the behavioral drivers of factor returns, and as human beings, I’m skeptical that many of these behavioral drivers are going away. Will we no longer be subject to the lottery effect? Will we no longer chase glory stocks at the expense of more mundane stocks? I doubt it.

We have seen certain factors getting arbitraged away in the past by regulation. We saw this in the 1990s with the estimate revision factor. It was a big focus of investors in the 1990s, but after the passage of Regulation FD in 2000, its performance has been spottier. There’s a good reason for that, because the information underlying that factor is now evenly distributed. Those types of changes are worth monitoring.

Hougan: How should investors evaluate smart-beta strategies?
Mazza: You want to borrow from the techniques people have used to evaluate active managers—the classic “five P’s”: people, philosophy, process, performance and price. But you need to use those P’s differently.

In the case of people, ask: What firm is sponsoring the ETF, and who is the index provider? For philosophy, ask whether there’s credible evidence that some particular premia—value, momentum, etc.—exists, and ask if it’s being measured appropriately. For process, you really want to dig into the index methodology and see if it makes sense. And importantly, you want to do all that before you get to performance or price.

With performance—if it’s real-world performance—ask yourself if it delivers what was intended by the portfolio’s design. With a new product, you can look at the backtest, but you’ll want to take a “trust but verify” approach, and look closely at the underlying methodology.

Finally, with price, you want to look at the total cost of ownership: the bid/ask spread, the expense ratio and the premium/discount variability.

And of course, overall, I’ll focus on how the product fits in a portfolio long term. That way, I can still be happy with it even if the short-term outcome goes against me.

OppenheimerFunds is not affiliated with Inside ETFs.