Is One Index Really Better Than Another?

Couch Potato investors try to capture the returns of entire asset classes rather selecting individual securities. And we do that with passively managed funds designed to track indexes that represent each of those asset classes. But which index, exactly?

Since Vanguard announced last fall that it would be ending its relationship with the index provider MSCI, the question is worth considering. Vanguard’s Canadian equity ETF, for example, was originally benchmarked to the MSCI Canada Index but now tracks the FTSE Canada Index. Can investors expect the fund to behave differently now?

The short answer is yes, but the full story is more subtle: if the past decade is any guide, the performance of the two indexes will vary from year to year. But the differences are likely to be the result of random noise, and neither of these benchmarks can be said to have any meaningful advantage over the other.

There’s more than one way to measure the market

Investors often refer to “the index” the same way they refer to “the dictionary.” But in both cases, the definite article isn’t really appropriate: there isn’t a single authority. Just as Oxford, Merriam-Webster and American Heritage all define words slightly differently, index providers use various methodologies to define an asset class. And even if they look only at cap-weighted indexes, the final products can vary a lot.

Here’s how MSCI and FTSE describe their Canadian equity benchmarks:

“The MSCI Canada Index is designed to measure the performance of the large and mid cap segments of the Canada market. With 95 constituents, the index covers approximately 85% of the free float-adjusted market capitalization in Canada.”

“The FTSE Canada Index [currently with 78 holdings] is a market-capitalization weighted index representing the performance of Canadian large- and mid-cap stocks. The market-cap-weighted FTSE Canada Index represents about 75% of the Canadian equity market.”

Although the above indexes mention mid-cap stocks as well as large caps, the more popular S&P/TSX 60 is also comparable, since it covers almost the same slice of the market as the FTSE benchmark:

“Designed to represent leading companies in leading industries, the S&P/TSX 60 covers approximately 73% of Canada’s equity market capitalization. Its 60 stocks make it ideal for large cap coverage and a cost-efficient way to achieve Canadian equity exposure.”

Different paths to the same place

You might think all three indexes would show virtually identical performance—and you’d be right if you look at their 10-year track record. But here are the year-by-year returns since 2003. The “Variance” column indicates the percentage-point difference between the best and worst performer in each year.

MSCI FTSE S&P Variance
2003 27.1% 27.8% 25.5% 2.3%
2004 13.8% 14.5% 13.8% 0.7%
2005 25.6% 26.6% 26.3% 1.0%
2006 17.9% 19.7% 19.2% 1.8%
2007 10.5% 11.4% 11.1% 0.9%
2008 -31.4% -31.8% -31.2% 0.6%
2009 33.6% 32.3% 31.9% 1.7%
2010 14.9% 12.5% 13.8% 2.4%
2011 -10.0% -9.9% -9.2% 0.8%
2012 7.5% 7.7% 7.9% 0.4%
10-year 9.20% 9.30% 9.24%

.

If you invested for the full 10 years, it would have made no meaningful difference which index you decided to track. But there would have been some significant variance along the way: how about 2.3% basis points underperformance of the S&P/TSX 60 compared with the FTSE benchmark in 2003, or the 2.4% outperformance of the MSCI index over its FTSE counterpart in 2010?

Now imagine you run an actively managed Canadian equity fund and you earn 13.9% in 2010. Did you beat the index? If your fund is benchmarked to the MSCI index you lagged by 100 basis points. But you outperformed the other two. Someone tell the marketing department there’s an idea here.

The takeaway message for investors is simple: short-term performance differences between similar indexes are meaningless. Sometimes the reason is one or two big movers happened to get more weight in one index than another. It might be as simple as the dates the indexes are rebalanced. Remember this when you’re comparing ETFs tracking the same asset class, whether it’s VCE and XIU in Canadian equities or comparable bond funds such as the BMO Aggregate Bond (ZAG) and the iShares DEX Universe Bond (XBB). If the benchmark indexes are very similar, ignore any differences in returns over one or two years and go with the one that charges the lowest fee.

9 Responses to Is One Index Really Better Than Another?

  1. Value Indexer June 24, 2013 at 10:45 am #

    Your marketing idea sounds similar to a funny comment in Rob Arnott’s book on fundamental indexing. He mentioned that if fundamental indexes really outperform traditional ones, lazy fund manager could run a “closet fundamental index” while benchmarking to the regular index so they could earn a bonus with no extra work.

    With ideas like that you could be a great mutual fund manager :)

  2. Patrick Doyle June 24, 2013 at 2:56 pm #

    @Value Indexer: right on about “fundamental indexes”. Personally I lump them together with technical analysis as pseudo-science until I see some evidence to the contrary.

    The whole concept of a “fundamental index” just begs the question. To believe in the existence of a workable fundamental index is to believe that the people who constructed the index have found a way beat the stock market systematically and enduringly. If they can do that, why not just start another Berkshire Hathaway and become a billionaire?

  3. Brad June 26, 2013 at 2:16 pm #

    Another factor in deciding if one index is better than another is how index revisions are made and how susceptible they are to front-running. Apparently this is part of the reason that Vanguard switched from MSCI to FTSE and CRSP. I’m not sure what FTSE does, but CRSP keeps index changes private rather than pre-announcing them.

    Front-running index changes is estimated to cost 21-28 basis points for the S&P500 and about double that for the Russel 2000. Note that this loss doesn’t show up as tracking error.

  4. Canadian Couch Potato June 26, 2013 at 2:37 pm #

    @Brad: This is a good point, and it’s one important reason I like total-market indexes: it’s extremely rare for there to be significant additions or deletions—unlike with the S&P 500. That said, index providers have made strides to reduce the impact of front-running, so it’s not as much of a problem as it used to be.
    http://canadiancouchpotato.com/2010/02/10/the-sp-500-effect/

    What is still a problem with the Russell 200o, especially, is that it is only rebalanced once a year, and in the intervals many small-cap stocks may become a mid-cap stocks but remain in the index for many months. That leads to an “impure” style index: that is, a small-cap index that includes more than just small caps.

    CRSP has created a method for smoothing out the transition between small, mid and large cap indexes. It’s described here:
    http://www.crsp.chicagobooth.edu/indexes/key_concepts_3.html

  5. Heather June 28, 2013 at 11:30 pm #

    Hi,
    I’m a novice Index funds investor with the TD e-series and I’m noticing that since inception most funds have been 0.0 or like the Nasdaq at -1.2 since inception in 1999.

    How often do you suggest I rebalance my portforlio with the e-series?
    Thanks

  6. Heather June 29, 2013 at 12:36 am #

    i have also just read your post on:
    Why Rebalance Your Portfolio?
    So I’m guessing there will be no clear answer to how often I should rebalance my portfolio but no what concerns me is how will I make $$$ with the couch potato portfolio when since inception US equities Index -0.2 Internationl Index 0.2%
    How are you supposed to profit from indexes that have performed so poorly since 1999?

    Is there something I am missing?

  7. Mellow July 1, 2013 at 1:23 pm #

    Hi,

    Interesting article and site! I’m wondering what makes you choose a particular country’s index to invest in? From most articles and websites I’ve read, the authors tend to invest in an index from their own country, is this purely to avoid FX risk, or is there some other reasoning behind this?

    I’m from the UK and I’m wondering if it would be best for me to invest in a FTSE100 tracker, or a US based tracker (e.g. Dow or S&P), or something else!

    Thanks!

  8. expers September 29, 2013 at 12:40 am #

    @Heather, I’ve wondered the same thing as I too am new to this and have gone with the e-series. If you look at calendar performance the majority of years are in the plus column, with 2008 and thereabouts causing a huge drop. What I’ve taken away from this is that now is a very good time to buy for the expected rebound which is already taking place. If you look at a historical chart of overall stock performance going back to the early 1900’s there is a persistent upward trend which is hard to deny. 1999 – 2013 is only 14 years and the 2008 recession was really quite enormous. If you are youngish therefore, it seems like it would make sense to invest a good portion of your money into stocks (e-series or etfs) because that 100+ year historical upward trend is impossible to deny, just be prepared to weather any similar 2008’s which may arise in the future. This is a reason why it’s advised for older investors not to expose themselves too heavily to stocks, because they won’t be alive long enough to see the bounce back.

  9. expers September 29, 2013 at 12:44 am #

    @Mellow, I would imagine its the FX risk, maybe a bit of patriotism and the comfort level of dealing with businesses which are part of your own world – bank stocks with banks you bank with, grocery store stocks with grocery stores you shop at, etc. Personally I prefer a little more romanticism to filter into my choices, for better or worse, I admire Japanese culture and history so invested a good portion into the td e-series japanese fund and it has done exceptionally well compared to the canadian fund (in the short term at least).

Leave a Reply