Tuesday, September 29, 2009

Statistical Ephiphanies: My Favorite

I've been thinking about statistics lately, and I'm having one of those probably-obvious epiphanies. So in statistics, you generally measure whatever you're interested in on what you hope is a representative sample, then you generalize to the population as a whole. The more representative your sample is, the more accurate your statistics are for the population as a whole. Assuming you are truly randomly selecting your sample, the best way to increase the ability of your statistic to generalize to the population is to increase your sample size. The bigger the sample, the more people there are to neutralize any weird flukes, the less noise there is, and the more accurate your statistic is.

What happens if you keep expanding your sample size? Eventually the sample you are polling/measuring/whatever is your entire population. If you're measuring the entire population, your statistic is entirely accurate. There isn't a margin of error. You have attained absolute truth. (Congrats.)

But wait. How much does measuring whatever quality of everyone on the planet actually tell you about humanity in any true sense? Can't the entire population be seen sort of as a sample of all possible human beings? (I would hazard a guess that it's not a very random one at all, but that's beside my point.)

Let's take my favorite example, gender differences in math. Now, I assume we have not tested the mathematical skills of every single human on the planet, but let's pretend we had and that the numbers were similar to those I've seen for various samples in the U.S. The median math score for males is about five percentage points higher than the median math score for females.

(The distributions are pretty wide and overlap almost entirely; only the very, very, very tip (a fraction of one percent) in either direction is occupied solely by males or females. This basically just means that yeah, maybe the top however-many math people are male and the bottom however-many are female, but since every single person I know is almost assuredly in the overlapping spots, there's not really much that can be said for any given man or woman about their relative abilities. For example, I am a girl; supposedly I am worse at math than boys. No randomly chosen guy should assume that means he is better at math than me, a (not-so-randomly-selected) girl. Assuming elementary-school scores on the ITBS and the percentiles they provide are accurate (probably not, but just for the sake of illustration), the fact that eight-year-old me was in the 99th percentile on math obviously means that eight-year-old me was better at math than a lot of eight-year-old boys. Even if the entire top 1% was boys (doubtful), that would still put eight-year-old me above 98% of eight-year-old boys taking the ITBS. (I feel like the math I just did was questionable, particularly since there's no guarantee that eight-year-old ITBS-takers were half boys and half girls, but, hey, there's a reason I'm not using more recent math scores...))

Anyway... So say we'd tested everyone on the planet and gotten a similar distribution. Hey, we all say, look, boys are indeed better at math! But what does that actually mean about masculinity making one good at math? Not necessarily anything. For one thing, correlation does not equal causation. Correlation on a planetary scale still does not imply causation. It would be rather impossible, not to mention highly unethical, to randomly assign people to a gender to see how that affected their math scores later in life.

And here's where my epiphany comes in. By sheer virtue of there being two groups, one of them basically has to be better and one worse on any given thing, be it math, communication, parking ability, cooking, singing, tennis, Tetris...whatever. Since there is a lot of variability in mathematical ability, like everything else, it is necessarily unevenly distributed regardless of how you divide the groups. It would actually be weirder if every division of humanity we could think of, whether by gender, race, religion, height, eye color, alphabetical order by middle initial, etc., scored exactly the same on every measure. It's like if you flipped a coin 100 times and got exactly 50 heads and 50 tails, and then did it again and got 50 and 50 again, just in a different order, then again, then again. Maybe if everyone on the earth vanished suddenly and God replaced them with six billion other random people, this time, girls would end up by a few percentage points just by virtue of which actual people were part of our group and this hypothetical group.

Of course, that's not to say that mathematical abilities don't actually have their basis in anything vaguely gender related. Obviously there are all sorts of things that influence mathematical ability, like what kind of society you're born into, what kinds of things your parents find important, how good your school was, whether anyone told you as a child that your group wasn't supposed to be as good at math as the other one, your specific math teachers, how good your parents were at math, what language you speak (yes, that's one reason Chinese kids are better at math than Americans, is because the words for numbers in Chinese make it easier to think about them and do things with them), whether your friends think math is cool, what else you're interested in, etc., etc., etc. Sure, gender might be one of them. Maybe testosterone is a math booster. (This might be slightly less unethical to test...) But I think it's important to realize that just because a majority of humanity is better at something than another group doesn't necessarily mean it's related to whatever the dividing characteristic is. Numbers are like that. There's variation. And most importantly, it's pretty obvious that all members of group X aren't better than all members of group Y at anything that I'm aware of, whether it's races and sports or genders and school subjects or species and some cognitive task.