The BMO Canadian Dividend ETF (ZDV), launched last month, is unusual among dividend ETFs in that it does not track an index. Instead, it follows a “rules-based methodology” to screen stocks according to yield, dividend growth and payout ratio.

Last week, I asked whether ETFs really need an index in order to play a role in a passively managed, low-cost and low-turnover portfolio. Instead of tracking a traditional third-party index, can an ETF’s fund manager simply draw up its own set of quantitative rules and accomplish the same thing? If I had to make that argument in court, I’d build my case this way:

Many indexes are not transparent. Let’s all agree that transparency is paramount in a passively managed fund. But we should acknowledge that many well-known indexes are not particularly transparent.

Few people question whether funds tracking the S&P 500 or the Dow Jones Industrial Average are passive. And yet, although the major criteria are public knowledge, the companies in the S&P 500 are selected by a committee. The DJIA is even more of a black box—there’s nothing at all transparent about how this granddaddy of indexes is built. Just because your ETF tracks a third-party index doesn’t mean that you can understand exactly how the holdings are determined.

The fund provider may be determining the strategy. In the olden days, index funds all tracked well-known third-party benchmarks, many of which had been around for decades. But many new ETFs track indexes that were commissioned by ETF providers themselves.

Claymore’s bond ETFs are a good example, as are several of BMO’s equity funds, including the BMO Dow Jones Canada Titans 60 (ZCN). Dow Jones has long managed a family of Titans indexes that track the large-cap market in various countries, but I don’t think it’s a coincidence that the index for Canada has 60 stocks (the same number as the S&P/TSX 60), nor that it was launched in May 2009, three weeks before ZCN started trading.

The point is that even if a third party creates the index, the ETF provider may still be the one making the decisions about the overall strategy.

Index licensing is expensive. Any time an ETF or index fund uses a third-party benchmark, it pays a licensing fee. Fund companies are tight-lipped about how large this fee is, but according to a recent Reuters article, it is typically one-tenth to one-third of the overall management fee. In most cases, that would be somewhere between three and 18 basis points.

When the ETF’s investing strategy is straightforward, you have to wonder whether it really makes sense to pay an index licensing fee. The BMO S&P/TSX Equal Weight Banks (ZEB) is the most glaring example: it holds equal amounts of the big Canadian banks, and that’s it. Does one really need to hire Standard & Poor’s to divide by six? Couldn’t the fund save its investors several basis points by getting rid of the index?

For what it’s worth, the index-free BMO Canadian Dividend ETF (ZDV) is currently the cheapest dividend fund in Canada, with an MER of 0.40%, including HST.

Indexes still have value

I’ve laid out these arguments because I want investors to make sure they understand the subtleties. I do think it’s possible for a fund to be transparent, passively managed and maybe even cheaper without a third-party index. But I also believe that an investor considering an ETF with no index needs to ask some pointed questions:

Is the methodology fully explained? Indexes don’t guarantee full transparency, but they certainly help: many providers do publish thorough explanations of how their indexes are built. The description of the stock-picking criteria used by the BMO Canadian Dividend ETF (ZDV), by contrast, is extremely vague. All we’re told is that stocks are chosen based on “the three-year dividend growth rate, yield, and payout ratio” and that “securities will also be subject to a liquidity screen process.”

By comparison, the index-tracking Claymore and iShares dividend ETFs use methodologies that are explained in much greater detail by S&P and Dow Jones, respectively. If BMO were to provide a more meaningful description of its strategy, and they could ensure that the manager is not at liberty to stray from these rules, then I would consider ZDV a passive fund. But we’re not there yet.

Who’s benchmarking the fund’s performance? The most useful measure of an index fund’s performance is its tracking error, or the difference between the returns of the index and the actual return achieved by the fund. The iShares Dow Jones Canada Select Dividend (XDV) returned 12.82% in 2010, compared with the index’s return of 13.32%. As an investor, I know that the tracking error of –0.50% is identical to the ETF’s fee, which means the fund was managed almost perfectly.

How will investors measure the performance of ZDV? Even if the fund is transparent about its methodology, it will be impossible to know how well the managers executed that strategy. An index provider does more than just draw up a methodology: companies such as S&P and Dow Jones also compute and publish the returns of their indexes monthly and annually, which holds ETFs accountable. There’s a lot of value there—certainly it’s worth paying a few basis points for that benchmarking.

I’m not ready to jettison the idea of passively managed funds that don’t track an index. But it’s too early to accept them uncritically. For now, at least, I believe there’s just too much potential for things to go wrong when no one is looking.