Why People Overestimate the Risk of Black Swan Events

In his 2007 book “The Black Swan: The Impact of the Highly Improbable,” finance writer Nassim Nicholas Taleb attempted to educate the public about the danger of rare, unusual events. This is obviously important in the world of finance — asset market crashes come infrequently, but have an enormous impact on investors’ wealth, and potentially on the economy. Taleb implies that human beings underestimate the risk of these so-called black swans — in fact, he started one hedge fund (which closed in 2005), and advised another, based on this thesis.

But how often do people low ball the odds of catastrophe? And is it possible that we might do the opposite, overestimating the risk of things like stock-market crashes, pandemics and wars?

The problem with rare events is that they’re almost impossible to predict by looking at the past. Usually, when we want to determine how likely something is, we rely on some variation of a basic, classic procedure. We take the frequency that the thing has happened in the past, and this becomes our guess of how likely it is to happen in the future. If Atlanta has had sunny days 74 percent of the time during the past 50 years, then our best guess for the probability of a sunny day in Atlanta is 74 percent. Usually, we bring in other data, like recent global weather patterns or seasonal variations, to help improve the estimate.

But those guesses come with a lot of uncertainty. The rarer something is — a volcanic eruption or an asteroid strike — the more hazy our projection about its probability becomes. If we only have a short period of history with which to formulate our guesses, the problem is even worse — imagine trying to predict the likelihood of a hurricane hitting Houston based only on the weather report from the previous week. And if we have no good underlying theory of why something happens — for example, nobody really knows what causes stock-market crashes — then our job of estimating the probability is basically hopeless.

This is a very big problem in investing. There are lots of rare events that matter a huge amount — market crashes, regime shifts and bond defaults. We don’t have good theories for why most of these things happen. So statistical analysis, though it can help a little bit, isn’t very effective in assessing the risk of disastrous financial-market events.

What do we do when we have to make a forecast, but the past offers little guidance? We rely on other things — our hunches, gut reactions and untested theories.

In 2007, Harvard economist Martin Weitzman showed how this could explain the high returns we’ve seen in the U.S. stock market during the past few decades. The phenomenon of mysteriously high investment returns, called the equity premium puzzle, has been studied by financial economists for many years, but Weitzman’s explanation is probably the most compelling. When people have only vague notions about the underlying forces driving the stock market, they will change these ideas frequently as events unfold. Those changing guesses represent a kind of risk, which lowers stock prices today, causing them to have a higher return in the future. Essentially, people avoid stocks because they’re afraid they don’t understand what’s going on in the market.

Weitzman shows how people can act like they believe in black swans even if no black swans are actually present. But his model also implies that even if black swans do exist, people will act as if these unusual events occur much more than the historical record indicates.

In other words, Taleb might be wrong — people might be overestimating, rather than underestimating, the risk of market crashes.

Some recent survey evidence indicates that this might be true. William Goetzmann, Dasol Kim and the Nobel-winning economist Robert Shiller looked at 26 years of survey data, and found that people consistently say they expect things like stock-market crashes and earthquakes to happen more frequently than they really do. This is exactly the opposite of what Taleb might predict.

Of course, survey research like this has to come with a caveat — people are probably not able to precisely report their own guesses about probabilities. But surveys have proven increasingly useful in predicting investor sentiment, so this evidence shouldn’t be discounted.

So what does this mean for investors? It says that simply betting that the market is understating the probability of a crash is unlikely to yield market-beating returns. People following the opposite advice — buying and holding stocks for long periods of time — have done much better over the long run than people who think stocks are consistently overvalued. Maybe the way to make money is acting as if black swans don’t exist, even if they do show up from time to time.

This column does not necessarily reflect the opinion of the editorial board or Bloomberg LP and its owners.