Understanding Survivorship Bias

Survivorship Bias describes the flaw of an observer exaggerating the "track record" of events by failing to understand the importance of randomness. It is a particular type of selection bias, where "failures" are excluded without the observer taking notice. The term is often used in financial discussions, and thus has bearing on this month's theme, but understanding the import of survivorship bias can help with many aspects of life.

To increase our understanding of the concept of survivorship bias let's start with a primer on randomness. One of the best books you could read as a guide on either topic is Nassim Nicholas Taleb's book, Fooled by Randomness. Taleb asks us to consider the proverbial room full of monkeys banging away on sturdy typewriters. If the number of monkeys was infinite, there is a virtual certainty that one will eventually bang out a perfect copy of the Illiad. Now that we have found this hero among monkeys, Taleb asks, would anyone be willing to invest their life savings on a bet that this same monkey would write the Odyssey next? Taleb's thought experiment points us toward the relevant financial question, how much can past performance (here the typing of the Illiad) be relevant in forecasting future performance?

What is particularly interesting when considering problems of statistical inference is that a little knowledge of probability is perhaps more dangerous than none at all. We know that it is very unlikely for someone to consistently perform well without doing something right. It seems like commonsense to give past performance its due. But what we have to keep track of, Taleb points out, is the role of randomness in the person's profession and the "number of monkeys" in operation. Sample size matters. If the Illiad writing monkey was one of five monkeys in the experiment, his achievement is astounding. If there are one billion to the power of one billion monkeys engaged, it becomes clearer that the output is the result of luck. You need to count the monkeys.

When studies of money managers are conducted, the analysis often only looks at successful monkeys. That is because the managers who perform below the market go out of business or have their funds shut down. What we are left with is an example of "survivorship bias": a too rosy view of past performance that is the result of failing to ask "what happened to the failures?"

How does this impact us on a more daily basis? Taleb asks us to consider the plight of Marc, a Harvard educated savant who went on to attend Yale Law School. He makes $500,000 in a good year, lives in a Manhattan Co-op and has a house in the country. Not only is Marc haggard from work and stress, but in the eyes of his wife, he's a failure. That's because her peer group has become the neighbor's in her upscale co-op and the parents at the exclusive private school where they send their kids. From a materialistic standpoint, they come in toward the bottom compared with this exclusive peer group. Didn't Marc get 1600 on his SATs? Isn't he as smart as the Wilson's next door, who have millions and don't even deign to notice Marc in the halls?

The problem here is Marc's wife is comparing him to some of the most successful people in the world. Were she to track his success against his high school cohort, he'd be in the top 99.5% of income earners. Compared even to his Harvard classmates, he'd be in the top 90%. Compared to his graduating class at Yale, he'd be in the top 60%. But Marc's wife is using the wrong distribution to derive a rank -- he might feel like a failure when his savings amounts to so little compared to the successful CEOs who live in their building and that might be a very real emotional cost to survivorship bias -- but by any reasonable measure, Marc is a success. Why does it feel like failure? Because Marc and his wife have chosen to live in a place that excludes failure. Manhattan co-op apartments are for survivors.

Another example of misapprehending survivorship bias comes from the popular book "The Millionaire Next Door". This book points out the paradoxical point that many of America's millionaires work mundane professions and live ho-hum lives. They owe their wealth, the authors claim, to diligent accumulation. The problem with this analysis however, is that it exclude all the diligent accumulators who failed to become millionaires. The authors failed to count the monkeys.

At the root of these problems, is a folk theory we seem to have about money and performance. We want to believe that they are related. What randomness makes us consider, is that most financial success is not due to a track record of results, but rather, luck.

Comments

[Daniel]

I loved Fooled By Randomness (thanks for recommending it to me). The second edition has an expanded preface where Taleb sticks to his randomness theory, but also agrees that being prepared to grab your random ship when it comes in has its role in success. An easy way to say this may be "the early bird gets the worm". But we shouldn't think that worms are of our making. Ready monkeys often get the random worms.

[Carrick]

This is fascinating stuff. It goes along with the idea sometimes referred to as "affluenza", the never-ending desire for more wealth and material goods. I think there must be many Marcs at every level of the socio-economic ladder. We would be well-served to remember the role randomness, luck, and failure play in our lives.

[cheap levitra]

I agree with you Carrick

[how to make penis bigger]

drugs are bad ..m'kay?

[Nike Shox CL]

So nice to read your perfect blog on the spare time. Your article brings me different kind of feeling about the literature.

[Puma Clyde]

Life is always constantly looking for opportunities to realize their own value! I read your blog see you realize the value of your own! But you also want to help more people realize the value they place!

[buy tadalafil generic]

Thanks for this post. The timing could not have been more perfect.

[fat burners]

I really liked "The Millionaire Next Door".

[Phen375]

I can't agree with one commentator. Drugs like Phen375 are just fine.

[Jordan Shoes 8]

In a continent where 1 out of 2 people now own a cell phone, mobile communications have the potential to dramatically transform life on the continent, from bringing bring hitherto marginalized communities into the mainstream, to improving adfhealth, education, social services and promoting social advocacy and political transparency.

[Coach Outlet Stores Online]

"I would have expected that this should contain a routine that accepts a starting url and then traverses the website and picks up the website hierarchy "dynamically" and construct the xml accordingly.
one is dangerous and the other is not?

[xl pharmacy]

I love Survivor it's one of the best t.v programs specially because of the games are so real and the participants have to get into different difficulties, you have to see the injuries they get during the reality.

[The North Face Outlet]

I like you on facebook and follow through google reader!