The "Expert on Experts" on Who You Can -- And Can't -- Trust in Today's Market

“Expert”. It’s a term that gets thrown around an awful lot in parts of our society, and the financial world is no exception. But in an interview with Money magazine, researcher Philip Tetlock warns not to trust someone just because they’re called an expert, and explains why so many of these supposed “experts” failed to see the market crash and economic crisis coming.

Tetlock, notes Money’s Eric Schurenberg, “is the world’s top expert on, well, top experts”. His two-decade study included nearly 300 academics, economists, policymakers and journalists, and detailed how more than 82,000 forecasts fared against real-world outcomes. The result: “We found that our experts’ predictions barely beat random guesses – the statistical equivalent of a dart-throwing chimp – and proved no better than predictions of reasonably well-read nonexperts,” Tetlock told Money. “Ironically, the more famous the expert, the less accurate his or her predictions tended to be.”

As for why so many experts got the recent market crash and economic crisis wrong, Tetlock explains that greed lulled corporate execs into complacency; they were making so much money that they had no motivation to look closer at things. “But hubris may have played a bigger [role],” he adds. “In this case the biggest source of hubris was the mathematical models that claimed you could turn iffy loans into investment-grade securities. The models rested on a misplaced faith in the law of large numbers and on wildly miscalculated estimates of the likelihood of a national collapse in real estate. But mathematics has a certain mystique. People get intimidated by it, and no one challenged the models.”

Tetlock does say some forecasters are better than others, and he found that the most important trait the top forecasters shared involved not what, but how, they thought. To explain, he uses a well known saying: “The fox knows many things, but the hedgehog knows one big thing”.

“The better forecasters were like … foxes: self-critical, eclectic thinkers who were willing to update their beliefs when faced with contrary evidence, were doubtful of grand schemes and were rather modest about their predictive ability,” Tetlock says. “The less successful forecasters were like hedgehogs: They tended to have one big, beautiful idea that they loved to stretch, sometimes to the breaking point. They tended to be articulate and very persuasive as to why their idea explained everything. … Foxes often qualify their arguments with ‘however’ and ‘perhaps,’ while hedgehogs build up momentum with ‘moreover’ and ‘all the more so.'”

Their measured approach means foxes aren’t as entertaining as hedgehogs, Tetlock says, and that means the media gravitates toward the hedgehogs. But if you’re looking for stable long-term performance, Tetlock says the fox is the one you want helping you with investment decisions.

Hedgehogs can get spectacular hits with certain predictions, but they also get lots of false alarms, Tetlock says. The media keeps going back to them, however, because they can make vague predictions that are difficult to prove wrong, or because they can explain errors away by saying that if only “X” had happened, they would have been right — which Tetlock says is a “ridiculous” type of reasoning.

Tetlock also offers advice on how to know if you’re becoming a hedgehog: “Listen to yourself talk to yourself. If you’re being swept away with enthusiasm for some particular course of action, take a deep breath and ask: Can I see anything wrong with this? And if you can’t, start worrying; you are about to go over a cliff.”

The expert on experts also mentions two forecasters who he views as good voices to listen to in this market: Moody’s chief economist Mark Zandi and Larry Summers, a top economic advisor to President Obama.

Send a Comment

Your email address will not be published. Required fields are marked *