How to Track Down Tracking Error

How do you measure the performance of an index fund? That sounds like a simple question, but it’s one I think many investors still have trouble with. Otherwise I wouldn’t routinely get questions like, “How can you recommend US and international index funds when they have performed so poorly over the last 10 years?”

If you’re an index investor, your goal is to capture as much of the market’s return as possible. And since investing is never free, that means you’ll likely trail the market indexes by 20 to 60 basis points annually. That may sound like it’s setting the bar low, but the evidence is overwhelming that the majority of investors do much worse, whether they use actively managed mutual funds or pick individual securities.

None of the TD e-Series index funds, for example, has ever beaten the market. Yet they continue to outperform the vast majority of their peers: four of the six core funds in the e-Series family were first-quartile performers over the last 10 years, while the other two were in the second quartile.

So let’s return to the question of how you measure an index fund’s performance. First, let’s agree that a passively managed fund cannot be held accountable for the returns of the market. That would be like blaming the thermometer if the weather isn’t to your liking. Rather, an index fund must be judged according to how closely it tracks its benchmark index. The measure of success, then, is low tracking error.

Fees are not the only cost

I like to think of tracking error as the most complete measure of an index fund’s cost. Most investors focus on MER, and this annual fee is certainly the place to start. But a fund with a low MER and a consistently high tracking error is still expensive. Whether you’re lagging the index because of fees or because of the manager’s inability to track the index doesn’t much matter in the end. Your returns suffer either way.

Of course, index funds and ETFs with low management fees have much smaller obstacles to overcome, so they tend to have lower tracking errors as a result. But that’s not always the case. Some funds have less obvious expenses, such as sampling error and currency hedging in international funds. ETFs that use derivatives—such as forward contracts, swaps and commodity futures—often have significant trading expense ratios (TERs), the lesser known cousins of the MER.

Some indexes, by their nature, are just harder to track. Cap-weighted equity indexes such as the S&P/TSX 60 and the S&P 500 are much easier to replicate than fundamentally weighted or equal weighted indexes, for example. Bond indexes may be impossible to mimic perfectly, so the skill of the manager is more important than investors may realize.

Where to track down tracking errors

How do you find a fund’s tracking error?  iShares is the only ETF provider that offers a useful tool: visit the website and click Tracking Error Chart in the Resources box at the top right of the screen. Then choose any ETF from the drop-down menu and select the time period you want to examine.

For BMO funds, you can visit the web page for the individual ETF, click the Prices & Performance tab, and then click Tracking Error Chart. But the graph simply overlays the performance of the index and the fund, without providing any numbers. That makes to tool rather useless.

Claymore and Horizons do not have tracking error tools on their websites.

The most reliable way to check a fund’s tracking error is to get its Management Report of Fund Performance. This document is published semi-annually and is available on the provider’s website or from SEDAR. Under the heading Results of Operations, the report will disclose the fund’s annual return and compare that to its benchmark.

The final 2010 reports appeared earlier this month, and I looked up the tracking errors of most major index funds and ETFs. I’ll post the results later this week.

10 Responses to How to Track Down Tracking Error

  1. Preet April 25, 2011 at 8:59 am #

    Thanks for the link. I agree that total tracking error is the best way to evaluate an index fund. On the institutional stage, and since the management fee is part of the tracking error, consultants and institutional investors focus as much on this as they do the fee. If someone charges a higher fee for an index mandate, but tracks the benchmark more closely, they get the business.

  2. Jason April 25, 2011 at 3:22 pm #

    Why are index funds/ETFs so reluctant to compare their performance with the benchmark? Claymore I believe compares its own commissioned index (“RAFI Index”) with another index instead of the fund!

    I think very rarely a fund/ETF can beat its index very slightly or at least make up some or all of the MER if it is allowed to lend a small portion of the securities it contains. I guess that’s where “other income” (Box 26 of T3 receipt) comes from in an equity ETF.

  3. Canadian Couch Potato April 25, 2011 at 3:36 pm #

    @Preet: Thanks for the comment. It’s always interesting to hear what goes on on the institutional side, as these managers understand the subtleties that the rest of us often miss.

    @Jason: I think part of the issue here is that some ETFs don’t track their indexes very well, so they don’t want to highlight that fact. However, most ETFs do a pretty good job in this respect. The bigger problem is that investors may not understand that an ETF that lags its benchmark a little bit each year is actually doing its job – we are so accustomed to thinking that beating the market is the goal.

    Claymore compares its fundamental indexes to their cap-weighted competitors simply to demonstrate that the RAFI strategy would have outperformed in the past. But you’re right, I don’t really care if the index would have outperformed if the fund itself doesn’t track the index well. And Claymore’s record here is mixed: it’s outstanding with their Canadian equity fund, and not so great with the US and international funds. But it is getting better.

    You are right that most ETF providers engage in securities lending to try to offset some of the MER (as well as to make a little profit for themselves). That’s one of the reason tracking errors are quite often a little smaller than the MER.

  4. Bill April 25, 2011 at 11:10 pm #

    Is SEDAR only for Canadian-listed funds? If so, where can I find the equivalent for NYSE listed ETF’s?

    Thanks for pointing this out … an area I haven’t really looked at! I noticed that for XIC, the stated goal is “to achieve a minimal difference between the fund’s return and the index return.”

  5. Canadian Couch Potato April 25, 2011 at 11:56 pm #

    @Bill: SEDAR is only for Canadian funds. I haven’t been able to find a central location to retrieve annual reports for US-listed ETFs, but they are usually available on each individual ETF’s web page. In the US they are not called Management Reports of Fund Performance; they’re just called Annual Reports.

    For example, you can visit the page for the Vanguard Total Stock Market ETF and click the link that says “View prospectus and reports.”

  6. Dennis Findlay April 26, 2011 at 6:36 am #

    Thank you for your article on tracking error. I was about to purchase a BMO ETF that tracks the DOW only to find that the tracking error was considerable. Instead, I added to my iShares S&P500 ETF (XSP) which has a much smaller tracking error related solely to the MER.

  7. James May 20, 2011 at 11:58 am #

    Hi,

    Whats a reasonable tracking error? Under 1%?

    Thanks!

  8. Canadian Couch Potato May 20, 2011 at 12:06 pm #

    @James: For most equity or bond funds, I would expect a reasonable tracking error to be the MER plus no more than 10 basis points or so. For large, well-established funds, I would expect the tracking error to be even smaller. For international equity funds, however, that’s probably unrealistic.

    Overall, I would aim for a portfolio that tracks its indexes at under 0.5%.

  9. Mike B June 6, 2011 at 12:53 pm #

    Hi guys – I am not sure if this is the place, but the SEC puts all their filings on a website like SEDAR called EDGAR. I never looked up ETF’s on there but you can find the companies 10K’s and other filings on the database, just like SEDAR.

  10. Marcelo August 1, 2013 at 3:07 pm #

    Using Ishare web site data when i calculated some ETF tracking error (standard deviation of daily diference returns) i located some spikes. Usually this representative diferences occurs on the last 2 days of the month, and the second movement neutralize the on neutralize the previous.
    Example:
    Index Return NAV Return Diference
    07/30/2007 -0,12% 1,34% 1,46%
    07/31/2007 0,61% -0,84% -1,45%

Leave a Reply