Is Your Smart Beta Strategy Doing Its Job?

This is Part 9 in a series about smart beta ETFs. See below for links to other posts in the series. In this installment, we look at how you can tell whether smart beta indexes will really perform as expected.


As we’ve worked through each of the factors targeted by smart beta ETFs—value, size, momentum, low volatility and quality—we’ve been careful to point out that no one really knows whether these premiums will persist in the future, especially once they’re on the radar of millions of investors.

However, we can look backward to see whether smart beta indexes have actually behaved as you would expect. Say you’re considering an ETF that targets value and small-cap stocks. If the index did well when value and small stocks outperformed, and did poorly when they lagged, that’s reassuring. But if you find the index’s performance had no correlation with value and size—or if it delivered outsized returns during periods when these stocks were dogs—that’s a red flag. It would be difficult to put a lot of faith in that ETF.

Think of a gardener who buys a system of rain barrels to capture the precipitation that falls on her roof. She knows she can never predict the weather next month, but if she can be confident the barrels will reliably capture a high percentage of any rain that does fall, that would be enough to justify the purchase. And if she finds the barrels full after two weeks of drought, that’s cause for concern, because she can’t explain where that water came from.

Turns out there is a way to put smart beta strategies to the test: it’s a statistical tool called regression analysis. Justin Bender has described the process in detail, but we can skip the gory details here and focus on the bigger picture. The general idea is to obtain the monthly performance data for each factor (value, size, quality and so on), according to its “academic” definition. Then you get monthly performance data for the fund you’re interested in, or its index. Finally, you compare these sets of data to see whether the strategy performed as you would expect.

Blitzing the data

Some of the most interesting work in this area comes from Dutch researcher David Blitz, summarized in a recent paper called Factor Investing with Smart Beta Indices. Using the method I’ve described above, Blitz examined popular smart beta indexes from Russell, MSCI and S&P to see whether they really captured the factors they targeted.

The results were mixed. The well-known Russell 1000 Value Index, for example, turned out to be “not very suitable for investors seeking pure and sizable exposure to the (academic) value factor.” Meanwhile, “the MSCI High Dividend index provides a huge (83%) exposure to the Low Volatility factor,” which suggests that high dividends are not really driving the returns of that strategy. There was water in the barrels, but it didn’t come from rainfall.

Overall, Blitz argued that investors will have a tough time selecting smart beta ETFs from the growing menu of choices. “The amount of factor exposure provided by popular smart beta strategies differs considerably,” he writes, and “these results imply that factor investing with smart beta indices is not as straightforward as one might think.”

The first dimension

Before we look at some specific smart beta strategies in this light, let’s discuss how to interpret the data. By definition, a traditional index fund has no additional exposure to any of the factors: it’s all “dumb beta.” So if you did a regression analysis on a plain-vanilla index ETF you should expect to get a value of 1 for the market factor (beta) and a zero for all the others. This would indicate that 100% of the fund’s performance can be explained by its exposure to the broad market:

Theoretical Broad-Market Index Fund
Market (beta) 1.00
Value 0.00
Size 0.00
Momentum 0.00
Quality 0.00

A good place to start our examination of smart beta strategies is Dimensional Fund Advisors, one of the pioneers in this area. Although DFA doesn’t offer ETFs, their mutual funds are designed to get “strong exposure to securities of small issuers and securities that it considers to be value securities.” They now also include an additional screen for companies with high profitability, which is closely related to what we’ve been calling the quality factor.

I asked Justin to run a regression on these indexes to see how much of their performance could be explained by this additional exposure to value, size and quality. We should expect to see a number much greater than zero for value and size, as well as a positive value for quality. Finally, any time you run a regression, you won’t be able to explain everything with factor exposure: in the table below this is the “Unexplained” category, and we want this number to be as low as possible.

With that mind, here are the numbers Justin got for two DFA indexes tracking Canadian and US stocks using 17 years of data from January 1999 to the end of 2015:

DFA Canadian Vector Equity Index
Market (beta) 0.98
Value 0.30
Size 0.17
Momentum -0.07
Quality 0.05
Unexplained 0.53%


DFA US Vector Equity Index
Market (beta) 1.07
Value 0.47
Size 0.29
Momentum -0.07
Quality 0.17
Unexplained 0.14%
Sources: Dimensional Returns 2.0, AQR Data Library

What do these numbers tell us? The relatively high numbers we see for value and size suggest the DFA indexes do a good job at “tilting” toward underpriced and small-cap stocks. The indexes got some additional exposure to the quality factor as well (more in the US than in Canada). Some of the performance is unexplained, but not a lot. Overall, when value, small-cap or quality stocks outperform (or lag) you should expect similar results from these indexes.

How do the iShares Multifactor ETFs compare?

Now let’s consider a different multi-factor strategy available to Canadian ETF investors. The iShares Edge MSCI Multifactor Canada (XFC) was launched in September 2015 along with similar funds for US and international stocks. These ETFs track the MSCI Diversified Multiple-Factor Indexes, which focus on value, momentum, size and quality.

During the same 17-year period (from 1999 through 2015) the Canadian version of this index reported an annualized return of 11.86%, compared with just 7.24% for the broad market. That’s some huge outperformance. But we want to know the reason: did the index really get added exposure to the four factors, and if so, did that explain the much higher return?

When Justin analyzed the Canadian data for a blog post he wrote in August, here’s what he found:

MSCI Canada IMI Diversified Multiple-Factor Index
Market (beta) 0.98
Value -0.03
Size 0.21
Momentum 0.07
Quality 0.07
Unexplained 2.41%
Sources: MSCI, AQR Data Library

This result is a head-smacker. The index did indeed have added exposure to small cap stocks, but its huge outperformance could not have come from this tilt, because over the 17 years in question small caps actually lagged the broad Canadian market. The added exposure would have hurt, not helped. There was a wee tilt to momentum and quality (0.07 is pretty modest), but the index actually had less exposure to value stocks than a traditional index fund would have had.

Bottom line: this smart beta index may have delivered market-beating returns when it was backtested, but it’s hard to understand why. Indeed, the amount of unexplained performance was huge at 2.41% per year. Any time you can’t explain the reasons a strategy outperformed in the past, it’s hard to have any confidence it will work in the future.

Other posts in this series:

Smart Beta ETFs: Your Complete Guide

A Brief History of Smart Beta

Understanding the Value Factor

Understanding the Size Factor

Understanding the Momentum Factor

Understanding the Low Volatility Factor

Understanding the Quality Factor

Exploring Multi-Factor Models


12 Responses to Is Your Smart Beta Strategy Doing Its Job?

  1. Derek C October 17, 2016 at 1:16 pm #

    Though not displayed in the data, the backtesting for the BMO low volatility ETF in Canada was also quite compelling. It has more than outperformed the higher MER since inception, but I wonder if anyone knows why.

  2. Canadian Couch Potato October 17, 2016 at 1:21 pm #

    @Derek: Unfortunately there is no obvious source for low volatility data. That’s why it is not included in the regressions above. For the other factors, data can be obtained for free from the excellent libraries maintained by Kenneth French and Andrea Frazzini/AQR:

  3. Nick de Peyster October 18, 2016 at 8:40 am #

    Sometimes one can get a surprising result in a Fama-French regression because the investment manager is using different factors than Fama-French. It shouldn’t be that way, but it can be … maybe not on size, but on the other factors absolutely.

    Nick de Peyster

  4. phil October 18, 2016 at 3:36 pm #

    🙂 now we’re getting into it, nice! It’s amazing how misleading funds intentions can be.

  5. Canadian Couch Potato October 19, 2016 at 7:55 am #

    @Derek: Update from Justin: The “betting Against Beta” (BAB) data in the AQR library is a proxy for the low volatility factor. When Justin ran a regression on the BMO Low Vol index, it had a significant tilt to this factor.

  6. jamie October 19, 2016 at 8:29 am #

    “the amount of unexplained performance was huge at 2.41% per year. Any time you can’t explain the reasons a strategy outperformed in the past, it’s hard to have any confidence it will work in the future.”
    Dan I’ve been looking back through this site trying to find the article which questions the superiority of cap weighted indexes. I’m grinning as I remember the description of monkeys producing a basket of stocks that beat the overall market. What are your thoughts? Is it just luck or is there a critical flaw in cap weighing–exposure to exuberance, lack of re-balancing…?

  7. Canadian Couch Potato October 19, 2016 at 8:59 am #

    @jamie: This may be the article you’re referring to:

  8. Matt October 19, 2016 at 9:42 pm #

    Was the BMO Low Vol analysis for the ZLB ETF or a different fund(s)?

  9. CharlieFox October 20, 2016 at 6:53 pm #

    I glanced at it last week on Google finance but don’t remember the exact numbers.
    But ZLB has like 1.4B worth of market cap, while VCN only has like 0.6B.
    Does that difference matter? Cause I remember reading somewhere that the bigger the fund gets, the less likely they’ll perform as well as before. So does that mean ZLB won’t do as well as before?

  10. Canadian Couch Potato October 20, 2016 at 8:03 pm #

    @Charlie: The size of an ETF should have nothing to do with its performance: if anything, the more scale it gets the lower its tracking error should be and the fewer capital gains it should distribute. My guess is that you read that certain strategies may no longer work once they are widely adopted, which may well be true. But this would have to happen on a huge scale (i.e. investors everywhere suddenly embraced stocks in a specific sector, or showed a huge bias toward dividends stocks, etc.), not on the scale of a single ETF.


  1. What we are reading on 10/18/2016 - UNDERVALUED STOCKS - October 18, 2016

    […] Is Your Smart Beta Strategy Doing Its Job? […]

  2. Weekend reading: The $35 billion passive man – Frozen Pension - October 22, 2016

    […] Is your Smart Beta strategy doing its job? – Canadian Couch Potato […]

Leave a Reply