If the evidence in favour of passive investing is so strong, why isn’t the strategy more popular? I hear that question all the time, and there are several answers, including effective marketing by investment firms and a general lack of awareness. But there’s another reason that affects even those who are well aware of the research. It’s the deep emotional appeal that comes from the possibility—however small it might be—of achieving market-beating returns.

I thought about this recently during a conversation with an investor who was considering moving from his current advisor (who used a highly active strategy) to an indexed approach. Robert had been with his advisor for more than 10 years, and it was clear his portfolio had lagged the indexes over that period. He also complained the advisor was providing no financial planning and no tax management: there was only active investment management, and it had failed for more than a decade. Why, then, did Robert find it so hard to break free?

As we spoke, the answer became clear: Robert wasn’t disputing that indexing had a higher probability of success. He just wanted to hold onto the possibility of outperformance.

“Might” makes right

This has always been a stumbling block for investors weighing the merits of active and passive strategies. If you make decisions based on a rational assessment of the evidence, it’s easy to come down on the side of indexing. But if people were always coolly analytical when making financial decisions, no one would ever walk into a casino or buy a lottery ticket. People who play the slots or buy scratch-and-win tickets are well aware their chances of winning are slim. But they’re not zero: you might win the jackpot. And as long as there is a possibility of winning, the games will hold enormous appeal.

In the same way, Robert was well aware that future outperformance was highly unlikely, but it wasn’t impossible if he stayed with his active advisor. With indexing, however, the probabilities would be in his favor but the possibility of earning market-beating returns would fall to zero. And it was awfully hard for Robert to accept that. It was like letting go of a dream.

Play the odds

Robert’s advisor seemed to understand this instinctively, and she exploited it skilfully. She showed him performance figures over carefully selected periods when her strategy did outperform (typically when she was sitting in cash during a down market). Then she showed him the impressive back-tested results of a new active strategy she was planning to use. She couldn’t explain why she hadn’t actually used the strategy in the past, but she didn’t have to. She just needed to dangle the possibility that it might continue to work.

I understand it can be hard to let go of the hope you’ll earn market-beating returns, but I encouraged Robert to look at it differently. For years he was paying for something that delivered negative value. If he wanted to keep using an advisor, why not accept close-to-market returns and pay for services that always have value, such as financial planning, tax management and long-term discipline? And if he wanted to invest on his own, why not enjoy the guaranteed boost he’d get from cutting his investment costs by about 75%?

Viewed in that light, a strategy based on high probability rather than remote possibility shouldn’t be depressing: it should be liberating.