How do you measure the performance of an index fund? That sounds like a simple question, but it’s one I think many investors still have trouble with. Otherwise I wouldn’t routinely get questions like, “How can you recommend US and international index funds when they have performed so poorly over the last 10 years?”
If you’re an index investor, your goal is to capture as much of the market’s return as possible. And since investing is never free, that means you’ll likely trail the market indexes by 20 to 60 basis points annually. That may sound like it’s setting the bar low, but the evidence is overwhelming that the majority of investors do much worse, whether they use actively managed mutual funds or pick individual securities.
None of the TD e-Series index funds, for example, has ever beaten the market. Yet they continue to outperform the vast majority of their peers: four of the six core funds in the e-Series family were first-quartile performers over the last 10 years, while the other two were in the second quartile.
So let’s return to the question of how you measure an index fund’s performance. First, let’s agree that a passively managed fund cannot be held accountable for the returns of the market. That would be like blaming the thermometer if the weather isn’t to your liking. Rather, an index fund must be judged according to how closely it tracks its benchmark index. The measure of success, then, is low tracking error.
Fees are not the only cost
I like to think of tracking error as the most complete measure of an index fund’s cost. Most investors focus on MER, and this annual fee is certainly the place to start. But a fund with a low MER and a consistently high tracking error is still expensive. Whether you’re lagging the index because of fees or because of the manager’s inability to track the index doesn’t much matter in the end. Your returns suffer either way.
Of course, index funds and ETFs with low management fees have much smaller obstacles to overcome, so they tend to have lower tracking errors as a result. But that’s not always the case. Some funds have less obvious expenses, such as sampling error and currency hedging in international funds. ETFs that use derivatives—such as forward contracts, swaps and commodity futures—often have significant trading expense ratios (TERs), the lesser known cousins of the MER.
Some indexes, by their nature, are just harder to track. Cap-weighted equity indexes such as the S&P/TSX 60 and the S&P 500 are much easier to replicate than fundamentally weighted or equal weighted indexes, for example. Bond indexes may be impossible to mimic perfectly, so the skill of the manager is more important than investors may realize.
Where to track down tracking errors
How do you find a fund’s tracking error? The most reliable method is to get its Management Report of Fund Performance. This document is published semi-annually and is available on the provider’s website or from SEDAR. Under the heading Results of Operations, the report will disclose the fund’s annual return and compare that to its benchmark.
Thanks for the link. I agree that total tracking error is the best way to evaluate an index fund. On the institutional stage, and since the management fee is part of the tracking error, consultants and institutional investors focus as much on this as they do the fee. If someone charges a higher fee for an index mandate, but tracks the benchmark more closely, they get the business.
Why are index funds/ETFs so reluctant to compare their performance with the benchmark? Claymore I believe compares its own commissioned index (“RAFI Index”) with another index instead of the fund!
I think very rarely a fund/ETF can beat its index very slightly or at least make up some or all of the MER if it is allowed to lend a small portion of the securities it contains. I guess that’s where “other income” (Box 26 of T3 receipt) comes from in an equity ETF.
@Preet: Thanks for the comment. It’s always interesting to hear what goes on on the institutional side, as these managers understand the subtleties that the rest of us often miss.
@Jason: I think part of the issue here is that some ETFs don’t track their indexes very well, so they don’t want to highlight that fact. However, most ETFs do a pretty good job in this respect. The bigger problem is that investors may not understand that an ETF that lags its benchmark a little bit each year is actually doing its job – we are so accustomed to thinking that beating the market is the goal.
Claymore compares its fundamental indexes to their cap-weighted competitors simply to demonstrate that the RAFI strategy would have outperformed in the past. But you’re right, I don’t really care if the index would have outperformed if the fund itself doesn’t track the index well. And Claymore’s record here is mixed: it’s outstanding with their Canadian equity fund, and not so great with the US and international funds. But it is getting better.
You are right that most ETF providers engage in securities lending to try to offset some of the MER (as well as to make a little profit for themselves). That’s one of the reason tracking errors are quite often a little smaller than the MER.
Is SEDAR only for Canadian-listed funds? If so, where can I find the equivalent for NYSE listed ETF’s?
Thanks for pointing this out … an area I haven’t really looked at! I noticed that for XIC, the stated goal is “to achieve a minimal difference between the fund’s return and the index return.”
@Bill: SEDAR is only for Canadian funds. I haven’t been able to find a central location to retrieve annual reports for US-listed ETFs, but they are usually available on each individual ETF’s web page. In the US they are not called Management Reports of Fund Performance; they’re just called Annual Reports.
For example, you can visit the page for the Vanguard Total Stock Market ETF and click the link that says “View prospectus and reports.”
Thank you for your article on tracking error. I was about to purchase a BMO ETF that tracks the DOW only to find that the tracking error was considerable. Instead, I added to my iShares S&P500 ETF (XSP) which has a much smaller tracking error related solely to the MER.
Hi,
Whats a reasonable tracking error? Under 1%?
Thanks!
@James: For most equity or bond funds, I would expect a reasonable tracking error to be the MER plus no more than 10 basis points or so. For large, well-established funds, I would expect the tracking error to be even smaller. For international equity funds, however, that’s probably unrealistic.
Overall, I would aim for a portfolio that tracks its indexes at under 0.5%.
Hi guys – I am not sure if this is the place, but the SEC puts all their filings on a website like SEDAR called EDGAR. I never looked up ETF’s on there but you can find the companies 10K’s and other filings on the database, just like SEDAR.
Using Ishare web site data when i calculated some ETF tracking error (standard deviation of daily diference returns) i located some spikes. Usually this representative diferences occurs on the last 2 days of the month, and the second movement neutralize the on neutralize the previous.
Example:
Index Return NAV Return Diference
07/30/2007 -0,12% 1,34% 1,46%
07/31/2007 0,61% -0,84% -1,45%