The science behind 'going with your gut'
Ignorance may be bliss, but partial ignorance—or at least ignoring a glut of information and going instead with instinct—may be the best way to make certain vital decisions. So goes the argument put forth by Shabnam Mousavi, an assistant professor at the Carey Business School and a fellow at Berlin's Max Planck Institute for Human Development. Mousavi, who has studied classical decision-making theory, recently co-authored with Planck's Gerd Gigerenzer, a Journal of Business Research paper titled "Risk, Uncertainty, and Heuristics." The gist of the paper is that, in business at least, many decisions fall into a gray area of uncertainty. While it's tempting to do all that's possible to reduce that uncertainty by applying sophisticated computer models, opting instead for simple rules of thumb— known in Mousavi's field as heuristics—can produce decisions that outperform complex algorithms in real-world situations.
It's clear that many business leaders believe there's a time to put big data aside when thinking through a problem. Mousavi's paper cites 2012 research that found "almost half of the managers ([defined here as] subscribers to Chief Executive magazine, CFO magazine, CFO Asia, and CFO Europe) consider their 'gut feel' an important or very important factor in making capital allocation decisions."
Whether data wonks or those who fly by the seat of their pants make better decisions may depend on the problem's category. Traditional decision-making theory considers dilemmas that involve certainty and risk. "Under certainty each action is known to lead to a certain outcome. Under risk all outcomes as well as the probabilities of each outcome are known," writes Mousavi. For decisions that fall into either of these categories, taking the long route and applying extensive statistical analysis can bear fruit. But what about problems involving uncertainty, where the probabilities of outcomes are unknown? Here, Mousavi, who has a PhD in statistics and economics, writes that "calculations can provide illusory certainty." The illusion, she says, is based on the fact that there's zero evidence that decision-making models involving certainty or risk translate to those pesky (and common) uncertainty problems. "To even use a risk model [in those situations] is absolutely useless," says Mousavi.
Which brings us back to heuristics as a better way to solve problems of uncertainty. Some economists and others frown on heuristics, such as psychologist and Nobel laureate Daniel Kahneman, who noted in his New York Times best-selling book Thinking, Fast and Slow that they lead to errors and biases in judgments. But Mousavi says that what heuristics need is more study rather than scorn. Her approach is to reverse-engineer an uncertainty problem, starting with the solution, noting whether it was effective, then seeing which heuristics were applied to come up with it. She focuses on so-called "fast-and-frugal heuristics" and lists a dozen that have received considerable study. These are more than shortcuts. They involve what she calls "tools from the adaptive toolbox," which in humans evolved as survival skills. These include "imitating the successful," "imitating the majority," and the "recognition" heuristic. For the latter, one exploits partial knowledge to make a decision from among alternatives of varying degrees of familiarity. In business, that might mean deciding to place a satellite office in a city you've repeatedly visited versus one you've never seen.
"The study of fast-and-frugal heuristics has shown that less effort can lead to more accurate judgments. Heuristic strategies use learned and evolved core capacities such as memory and recall. This is why they are fast," Mousavi writes. The snap judgments that come from these core capacities may deliberately ignore other available information— "That's called irrational in standard decision-making theory," notes Mousavi—but these near-instantaneous judgments and decisions may nonetheless be correct.
To better understand heuristics and why we use them, Mousavi is going places that statisticians usually fear to tread, delving into fields that include psychology, sociology, and anthropology for clues. "Now we're using psychological insights and foundations to see what people are thinking, what beliefs they are holding, and what kind of information they're drawing on when they make decisions," she says. The real question is whether she can parse the best time to use a heuristic and whether that information can be modeled and taught. Mousavi says she thinks this can be accomplished; she has a book in the works.