The Survival Bias

I happen to be from India and find myself living on the East Coast. On numerous occasions random acquaintances have told me how they think Indians are unusually intelligent et cetera. This is, of course, based on their experience of Indians in the West. I like to point out that this is purely a survival bias. It just so happens that almost all Indian immigrants who come here are from the very top of the Indian cognitive distribution. Simply due to the barriers to entry, those who make it into graduate programs or get hired or otherwise make it to the West are far above the national average in cognitive ability. Since cognitive ability is at least partly inherited, this holds for second and third generation immigrants as well.

One might even speculate that this might account for the differentials in IQ test scores between Jewish people, Asians, Whites and Blacks in the U.S. The comment this post began with applies equally well to other Asian immigrants. We know that many of the top Jewish physicists and intellectuals fled to the United States from Europe after Hitler was elected. Might this be a more general phenomena? Perhaps those who made it across (and were therefore more likely to survive), were smarter on average than those who could not make it or chose not to? [Update:  Samsara points out that the Russian pogroms et cetera had the same effect, and that this is commonly understood among Jewish communities (especially those less prone to hubris).]

The standard deviation differential between Whites and Blacks in the United States also looks more explicable in light of the survival bias. Did the smart West Africans manage to flee and hide from the slave traders better, so that the average of those who were caught and transported was lower? A priori, the differential ought to be the other way around since there is a lot more genetic variation in Africa than in the rest of the world.

This notion actually generalizes quite a bit. Nassim Taleb points out a couple of very interesting examples. If you start swimming on a regular basis will you get the swimmers’ body soon? Isn’t it more likely that those born with a natural tendency to develop (or already have) a swimmers’ body become better swimmers??

If we measure the cognitive ability of convicted felons and find that it is lower on average than the population mean, does that mean we have evidence that ‘criminals are dumber’? No, of course not. It is far more likely that those who avoid getting caught are on average smarter than those who do. So criminals as a whole may be smarter or dumber than the population on average. 

Epidemiologists get this. So if one city, say New York, has a stronger campaign to exterminate rats, than another city, say Boston. Then, we will find the rats to be on average fitter and more adept at urban survival in New York than in Boston. Should we attribute that to NYC/Boston habitat differentials?

Along with other issues, this may go some way into explaining the decline effect. Some researcher reports finding a spurious effect which happens to be replicated by some others by sheer luck. Everyone hails the recent findings. Eventually, the luck will run out and the effect would be seen to disappear.

The same is true of fund managers’ ability to generate excess returns. If we start with a thousand fund managers facing stochastic returns, pure randomness dictates that some will have 5 year, 7 year, even 10 year runs of excessive returns. The cohort will still likely average the market return. If you only look at those who have been doing brisk business for 3 or more years you are bound to find them generating returns in excess of the market.

[Update: An old roommate and now a professor of financial economics sent me this paper by M. Carhart, published in 1997. It finds that the survival bias explains almost all persistence in fund returns.]

It does not hurt to think carefully.


5 thoughts on “The Survival Bias

  1. I agree and would like to add that many European Jews fled to America and the Middle East decades before Hitler and the Holocaust. The “mass exodus” of Jews from Europe actually began in the late 1800s/early 1900s. You could make the argument that these were “smart” Jews, who knew that trouble was coming (how my grandparents liked to phrase it). The ones who traveled the long and arduous distance to America and Palestine were those with more cunning, resources, and bravery than those who stayed (and perished).

    1. Yeah, absolutely. The Russian pogroms certainly played the same role. I find this a much more satisfying explanation than Ethnic differentials – which are likely positively correlated with the size and variation in the gene pool.

    2. @Dylan Smarter, or just more neurotic? 😉 All we know from these “mass exodus” anecdotes is that the survivors’ sensitive threat-detection mechanism picked up on a real and major threat at one point in their lifetime, and the ones with the resources to act on their instincts managed to get away. We don’t know what the rest of their lives were like.

      From an evolutionary perspective, a quick-trigger flight response is only a net positive in those rare occasions when it actually helps you flee something lethal. The rest of the time, it can be a costly and maladaptive response – hours wasted worrying about phantom threats, a higher risk of Type-I errors, and perhaps even an untimely death due to heart attacks and the like.

      As for whether this trait correlates with general intelligence, I’d love to see some actual data on that 😉

  2. I think the idea is that those who were ingeneous enough managed to flee. It’s not just a matter of fear and flight. From the perspective of the point of arrival, it’s sufficient that there were significant barriers to entry, so that individuals with higher cognitive ability are respresented at a higher rate among the arrivals than in the population left behind.

    In a toy model with high types and low types, the high types need just a sixth advantage or 7:6 odds in their favor for making it across in safety, in order for us to observe a one standard deviation differential. This makes it credible.

    All this is not to deny that this is highly speculative. You are right to be skeptical but there is a coherent argument for this theory.

    1. Ah – I was all about to argue that the only barrier or filter being applied to the immigrating population was the challenge of adapting to life on a new continent, but then I remembered the IQ testing that went on at Ellis Island. There’s your barrier to entry right there…in which case you’re absolutely right, this would indeed result in a visibly filtered population on the other side.

      That said, in a more general threat situation where there are no artificial filters being applied to the surviving population, I still believe you wouldn’t see any differential in general cognitive ability. It’s not the higher-order hardware that saves your ass in a serious emergency, it’s the stuff closer to your midbrain and spinal cord that will make or break you.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s