The Will Rogers Phenomenon

In statistics, the Will Rogers Phenomenon, named after the 1920s comedian who made that jibe, occurs when moving some element from set A to set B increases the average (mean or median) of both A and B.

It sounds impossible, but can actually happen quite easily. For example, A={4,5,6} & B={1,2,3}. If 4 migrates from A to B it’s pretty clear that both averages increase.

This has caused a certain amount of confusion over the years, for example in medicine. In this paper, doctors were trying to understand what had happened to cause the average life-span of certain groups of cancer patients to rise. The answer appears to be that a change in diagnostic technology led to a Will Rogers situation.

Oversimplifying considerably, patients were classified as “slightly ill” or “very ill”. Improvements in diagnoses then meant that some people who would previously have been classified as “slightly ill” were instead picked up as being “very ill”. This increased the average lifespan of the “slightly ill” group, as it were, by removing its illest members. But it also increased that of the “very ill” group because the new members were not as ill as those diagnosed “very ill” under the old system.

The confusing thing, of course, is that both groups see their average lifespan rise – which sounds wonderful. But it’s something of a mirage, because this could all happen without any individual seeing a longer life expectancy.