How often are supposedly “expert” forecasters’ predictions right? A lot less often than you might think, according to researcher Philip Tetlock. But in a recent interview on Barry Ritholtz’s Masters in Business podcast, Tetlock says you can take steps to make yourself a better forecaster.
Tetlock, a professor at the University of Pennsylvania and author of multiple books on the science of prediction, has found that as a group, supposedly expert forecasters don’t have much more predictive ability than average people. He’s also found that both experts and nonexperts tend to lag simple computer models when it comes to predictive power.
But his research also found that the best forecasters can teach us a lot about prediction. Those forecasters tend to be “foxes” rather than “hedgehogs”, a dichotomy based on an idea that dates back to the Greek poet Archilochus. Foxes tend to be self-critical, eclectic thinkers who adjust their beliefs when faced with contrary evidence and are modest about their ability to predict events; hedgehogs tend to have one big idea that they narrowly focus on, but can be very persuasive and articulate when speaking about it.
Tetlock says that his extensive research has found that foxes tend to be better forecasters. The trouble is that, because hedgehogs have big ideas and bold predictions, the media often gives them more airtime than measured, thoughtful foxes, Tetlock says. He talks about how inaccurate forecasters are able to deflect criticism about their mistakes and continue to offer often faulty predictions.
Tetlock also talks about how breaking down large questions into component parts can help in predicting things that seem unpredictable. And he says “superforecasters” can enhance their abilities by working with other superforecasters. Research shows that teams of ordinary forecasters tend to do about 10% better at predicting things than the “crowd”. “Prediction markets” do about 20% better than those teams, and teams of superforecasters do 15% to 30% better than prediction markets. Superforecasters working in teams even beat intelligence analysts that had access to classified information in a government-sponsored prediction contest, Tetlock says.
