Survivorship Bias describes the flaw of an observer exaggerating the "track record" of events by failing to understand the importance of randomness. It is a particular type of selection bias, where "failures" are excluded without the observer taking notice. The term is often used in financial discussions, and thus has bearing on this month's theme, but understanding the import of survivorship bias can help with many aspects of life.
To increase our understanding of the concept of survivorship bias let's start with a primer on randomness. One of the best books you could read as a guide on either topic is Nassim Nicholas Taleb's book, Fooled by Randomness. Taleb asks us to consider the proverbial room full of monkeys banging away on sturdy typewriters. If the number of monkeys was infinite, there is a virtual certainty that one will eventually bang out a perfect copy of the Illiad. Now that we have found this hero among monkeys, Taleb asks, would anyone be willing to invest their life savings on a bet that this same monkey would write the Odyssey next? Taleb's thought experiment points us toward the relevant financial question, how much can past performance (here the typing of the Illiad) be relevant in forecasting future performance?
What is particularly interesting when considering problems of statistical inference is that a little knowledge of probability is perhaps more dangerous than none at all. We know that it is very unlikely for someone to consistently perform well without doing something right. It seems like commonsense to give past performance its due. But what we have to keep track of, Taleb points out, is the role of randomness in the person's profession and the "number of monkeys" in operation. Sample size matters. If the Illiad writing monkey was one of five monkeys in the experiment, his achievement is astounding. If there are one billion to the power of one billion monkeys engaged, it becomes clearer that the output is the result of luck. You need to count the monkeys.
When studies of money managers are conducted, the analysis often only looks at successful monkeys. That is because the managers who perform below the market go out of business or have their funds shut down. What we are left with is an example of "survivorship bias": a too rosy view of past performance that is the result of failing to ask "what happened to the failures?"
How does this impact us on a more daily basis? Taleb asks us to consider the plight of Marc, a Harvard educated savant who went on to attend Yale Law School. He makes $500,000 in a good year, lives in a Manhattan Co-op and has a house in the country. Not only is Marc haggard from work and stress, but in the eyes of his wife, he's a failure. That's because her peer group has become the neighbor's in her upscale co-op and the parents at the exclusive private school where they send their kids. From a materialistic standpoint, they come in toward the bottom compared with this exclusive peer group. Didn't Marc get 1600 on his SATs? Isn't he as smart as the Wilson's next door, who have millions and don't even deign to notice Marc in the halls?
The problem here is Marc's wife is comparing him to some of the most successful people in the world. Were she to track his success against his high school cohort, he'd be in the top 99.5% of income earners. Compared even to his Harvard classmates, he'd be in the top 90%. Compared to his graduating class at Yale, he'd be in the top 60%. But Marc's wife is using the wrong distribution to derive a rank -- he might feel like a failure when his savings amounts to so little compared to the successful CEOs who live in their building and that might be a very real emotional cost to survivorship bias -- but by any reasonable measure, Marc is a success. Why does it feel like failure? Because Marc and his wife have chosen to live in a place that excludes failure. Manhattan co-op apartments are for survivors.
Another example of misapprehending survivorship bias comes from the popular book "The Millionaire Next Door". This book points out the paradoxical point that many of America's millionaires work mundane professions and live ho-hum lives. They owe their wealth, the authors claim, to diligent accumulation. The problem with this analysis however, is that it exclude all the diligent accumulators who failed to become millionaires. The authors failed to count the monkeys.
At the root of these problems, is a folk theory we seem to have about money and performance. We want to believe that they are related. What randomness makes us consider, is that most financial success is not due to a track record of results, but rather, luck.
