Simpson's paradox

First half: player hits bats avg. Ruth 28 100 0.280 *** Gehrig 54 200 0.270 Second half: player hits bats avg. Ruth 38 200 0.190 *** Gehrig 18 100 0.180

Two baseball players, Lou Gehrig and Babe Ruth, are compared during the first and second halves of the season.

Ruth had the highest batting average for both the first half of the season and the last half of the season. Who had the higher batting average for the entire season (i.e., both halves put together)?

Note: Based on "Lou Gehrig disease", and "hasty generalization", some coaches would tell players not to overdo it as they might get "Lou Gehrig disease" if they played too many games over time.

The first and second half statistics are as follows.

First half: player hits bats avg. Ruth 28 100 0.280 *** Gehrig 54 200 0.270 Second half: player hits bats avg. Ruth 38 200 0.190 *** Gehrig 18 100 0.180

Ruth had the highest average during both the first and the second halves of the season.

Who has the best overall batting average?

First half: player hits bats avg. Ruth 28 100 0.280 * Gehrig 54 200 0.270 Second half: player hits bats avg. Ruth 38 200 0.190 * Gehrig 18 100 0.180	Overall: player hits bats avg. ------ ---- ---- ----- Ruth 66 300 0.220 Gehrig 72 300 0.240 ***
Ruth had the highest average during both the first and the second halves of the season.	Gehrig had the highest average for the season.

This paradox is called Simpson's Paradox, after a statistician and not Homer Simpson. Simplified reason: comparing averages of averages.

[Berkley sex discrimination suit, baseball strike]

A paradox is a situation that seems to defy intuition.

It is possible to do any number of surveys, each of which indicates a certain conclusion (e.g., that there is more of a perceived bias in the South), and yet, when the data for all surveys are added together, the conclusion can switch (e.g., that there is more perceived bias in the North). Where is the problem?

Rule #1: Be very careful when comparing averages. Never add, multiply, or average averages. Averages are a measure of central tendency, or how grouped a set of numbers are, in which the relative numbers of each sample have been factored out.

Rule #2: Make sure that differences are statistically significant.

This paradox is called Simpson's paradox (from a 1951 paper by E. Simpson), although the paradox itself is older.

Simpson's paradox is not related to the cartoon character Homer Simpson.

Judea Pearl was the winner of the 2011 ACM Turing Award for "For fundamental contributions to artificial intelligence through the development of a calculus for probabilistic and causal reasoning ".
Judea Pearl is the father of journalist Daniel Pearl, who was kidnapped and murdered by terrorists in Pakistan connected with Al-Qaeda and the International Islamic Front in 2002 for his American and Jewish heritage. Wikipedia (as of 2023-03-25)

More: Converse fallacy: If A then B does not mean If B then A

More: Judea Pearl

In fact, in an actual job discrimination survey, a small but statistically significant margin of error arrived at the same result as our assignment. That is, two surveys concluded job discrimination against woman. Added together, the surveys indicated job discrimination against men. And the result was statistically significant.

The classical case demonstrating Simpson's paradox took place in 1975, when UC Berkeley was investigated for sex bias in graduate admission. In this study, overall data showed a higher rate of admission among male applicants, but, broken down by departments, data showed a slight bias in favor of admitting female applicants. Judea Pearl The explanation is simple: female applicants tended to apply to more competitive departments than males, and in these departments, the rate of admission was low for both males and females. Judea Pearl

Simpson's paradox first noticed by Karl Pearson in 1899, can be stated as follows: Every statistical relationship between two variables may be reversed by including additional factors in the analysis. (Judea Pearl, at http://bayes.cs.ucla.edu/jp_home.html) (as of 1974)

Simpson's paradox, first noticed by Karl Pearson in 1899, concerns the disturbing observation that every statistical relationship between two variables may be REVERSED by including additional factors in the analysis. http://bayes.cs.ucla.edu/jp_home.html (as of 1974)

For example, you might run a study and find that students who smoke get higher grades, however, if you adjust for AGE, the opposite is true in every AGE GROUP, namely, smoking predicts lower grades. If you further adjust for PARENT INCOME, you find that smoking predicts higher grades again, in every AGE-INCOME group, and so on. http://bayes.cs.ucla.edu/jp_home.html (as of 1974)

Imagine your family physician saying, "This drug seems to work on the population as a whole, but it has an adverse effect on males ... and an adverse effect on females." Only when you look at the numbers and agree to interpret the phrase "seems to have an effect" as a statement about a change in proportions do you begin to see that the calculus of proportions clashes with our intuitive predictions. Pearl, J. (1988). Probabilistic reasoning systems, 2nd revised printing. San Francisco, CA: Morgan Kaufman., p. 496. Equally disturbing is the fact that no one has been able to tell us which factors SHOULD be included in the analysis. Such factors can now be identified by simple graphical means. http://bayes.cs.ucla.edu/jp_home.html (as of 1974) Another example of Simpson's paradox is "reverse regression". In salary discrimination cases, should one compare salaries of equally qualified men and women or compare qualifications of equally paid men and women? Remarkably, the two choices led to opposite conclusions. It turned out that men earned a higher salary than equally qualified women, and SIMULTANEOUSLY, men were more qualified than equally paid women. (Judea Pearl)

Surveys (and petitions, etc.) tend not to be used to make decisions.

Instead, managers often tend to request the surveys and data that will support the decision they have already decided they want to make. (And ignore the results if the results do not fit what they expect or want).