When fears are high, it’s natural to want to jump out of stocks. But while it might seem logical to avoid equities during tough times, The Motley Fool’s Morgan Housel says history shows otherwise.

“Building wealth over a lifetime doesn’t require a lifetime of superior skill,” Housel writes (h/t The Big Picture). “It requires pretty mediocre skills — basic arithmetic and a grasp of investing fundamentals — practiced consistently throughout your entire lifetime, especially during times of mania and panic.”

Housel looks at three hypothetical investors: one who knows little about investing has put $1 in the S&P 500 every month since 1900; another — a “CNBC addict” looking to protect themselves from short term trouble — has done the same but has stopped investing as soon as a recession begins and then has deployed all of their built up cash as soon as the recession has ended; and a third — a fund manager concerned only with looking right in the short term — does the same $1 per month but stops investing six months after a recession starts, and deploys their cash six months after the recession ends.

Who wins? The one who seems to know the least — investor number 1. They would  now have more than $270,000, compared to $160,000 for the CNBC addict and $146,00 for the fund manager. The reason, Housel explains, is that most of the market’s gains are concentrated in a small number of days. Five percent of days made up for more than half of the difference between the dollar cost averaging investor and the recession-avoiding investor over that 113-year period, in fact. And, most of the days that matter come during periods of “sheer terror” — the market’s best 20 days ever all occurred either during the Great Depression, after the 1987 crash, or during the 2008 financial crisis.

Try to avoid losses, Housel says, and you miss gains. “It’s just buying and waiting,” he says of investing. “Most of what matters as a long-term investor is how you behave during the 1% of the time everyone else is losing their cool.”