OSU quiz–Campus Reform

A few days ago, there was a flap over an erroneous question on a quiz given to students at Ohio State University. I want to make two points:

1. None of the answers to the quiz question are correct. The suggested correct answer is not even the most likely to be true among the set of wrong answers.

2. Atheists do score slightly higher than Christians on IQ-derived vocabulary tests (the equivalent of about 2.9 IQ points in the 2006-2012 General Social Surveys) and on IQ-derived analogy tests (the equivalent of about 3.8 IQ points in the 1996 General Social Survey). But social psychologists ignore differences that are as large or larger perhaps because they do not fit their political narratives.

The Quiz

According to Campus Reform (noted at Daily Caller), an Ohio State University psychology quiz asked the following question:

Theo has an IQ of 100 and Aine has an IQ of 125.
Which of the following statements would you expect to be true?

• Aine is an atheist, while Theo is a Christian. [CORRECT]
• Aine earns less money than Theo.
• Theo is more liberal than Aine.
• Theo is an atheist, while Aine is a Christian.

Although all four answers are false (i.e., none would be expected to be true), according to reports and Campus Reform’s screen grab above, the quiz treated the first answer as true.

Even if atheists score 3-4 points higher on IQ tests than Christians, there are so many more Christians in the population that it is much more likely that someone with a 125 IQ score is a Christian than that such a person is an atheist.

To illustrate the quiz’s error in mathematical reasoning, consider this more extreme example:

You meet a guy named John, who weighs over 300 pounds. You know that most NFL All-Pro offensive linemen weigh over 300 pounds. So, following the reasoning of the Ohio State quiz, you would conclude that John is probably an All-Pro offensive lineman. Nonsense! Only a tiny percentage of men who weigh over 300 pounds are All-Pro linemen.

Likewise, though most NBA all-stars are tall, most tall people are not NBA all-stars. And while most atheists score very slightly higher on IQ tests than the rest of the population, most fairly high scorers on IQ tests are not atheists.

Someone who gets a 125 IQ score is less likely to be an atheist than a Christian, given that there are so many more Christians in the US. On the 2006-2012 GSS, for example, a 125 IQ (assuming a mean of 100 and a std. dev. of 15) would correspond to getting 9 questions correct out of 10 on an IQ-derived vocabulary test.

In the GSS, those with 9 questions correct are about 68% Christians (over 2-to-1 odds in favor of being Christian) and 24% atheists (over 3-to-1 odds against those with no religion). Thus in the IQ-derived GSS vocabulary tests, the odds that a person with a score equivalent to a 125 IQ is a Christian is about 6.5 times higher than the odds that the person is an atheist.

If one looks at the score on the GSS analogies test the last time it was used in 1996, the pattern is even stronger. Only 8% of those with a score of 13 out of 16 (corresponding to about a 125 IQ) were atheists, compared to 83% being Christians. Thus in the IQ-derived GSS analogy tests, the odds that a person with a score equivalent to a 125 IQ is a Christian is over 57 times higher than the odds that the person is an atheist.

The Sad State of Social and Political Psychology

There appears to be bias in the reporting of the results of IQ scores by social psychologists. When British longitudinal data are used, for example, it’s common to ignore the two best measures of who is a conservative that are in the databases used: (a) the British left-right scale, and (b) the variable self-reporting as a “conservative,” which is, after all, an actual party in the UK. If one uses either definition of conservatism, conservatives in the UK samples receive IQ scores slightly higher than Labour and slightly higher than average (advantages that are not always statistically significant).

Turning to the United States, when General Social Survey data are used to determine vocabulary scores, the Republican advantage over Democrats is typically ignored by social and political psychologists. Republicans score the equivalent of 3.4 IQ points higher than Democrats on the 2006-2012 GSS vocabulary tests and 3.7 IQ points higher on the 1996 analogy tests—about the same as the atheist-Christian difference.

And the liberal-conservative variable is usually analyzed in such a way that the researcher fails to report that conservatives are typically the middle IQ group between liberals and moderates. Further, in most (but not all) years, conservatives do not score significantly lower than liberals on the vocabulary test. In fact, the political group that scores the highest on the 2006-2012 GSS vocabulary tests is conservative Republicans (who score insignificantly higher than liberal Democrats).

Among religions in the GSS, the biggest differences in vocabulary scores are not between atheists and Christians, but between Jews and Muslims (though the Muslim sample is very small). Could you imagine an Ohio State quiz asserting that finding? Further, the 2.9 point IQ-equivalent score advantage of atheists over Christians on GSS vocabulary tests is much smaller than the 13.2 point advantage that Jews have over atheists. But that wouldn’t advance the more typical academic narrative about how smart atheists are.

While Christians average 5.89 questions correct (out of 10) on the 2006-2012 GSS vocabulary tests, respondents from Ohio’s region of the country (East North Central) score almost the same: only 5.88 correct. Can you imagine an Ohio State quiz assuming that it was probably true that Aine (125 IQ) was from Iowa (in the higher-scoring West North Central region) while Theo (100 IQ) was from Ohio (in the lower-scoring East North Central region)?

The Quiz Revisited

If one had to choose the likeliest among the unlikely options in the Ohio State quiz, probably the one most likely to be true is the third option, that “Theo [IQ: 100] is more liberal than Aine [IQ: 125].” I estimate this being true in the 2006-2012 GSS vocabulary tests about 35% of the time and in the 1996 GSS analogy tests about 45% of the time.

This compares to only a 19% probability in the GSS vocabulary tests both that a 125-equivalent vocabulary score (9 of 10 correct) was earned by an atheist and that a 100 IQ-equivalent score (6 of 10 correct) was earned by a Christian.

Conclusion

Though atheists on average score the equivalent of a relatively trivial 3-4 points higher than Christians on IQ-derived vocabulary and analogical reasoning tests, most people who earn scores equivalent to an IQ score of 125 are Christians. To assume that people with moderately high IQ scores are probably atheists reflects confused statistical reasoning and probably academic bias.

The obvious embarrassment for THE Ohio State University is mitigated by the fact that everyone makes mistakes, and most of us professors have drafted erroneous questions at some point in our careers.

NOTES:

1. Disclosure: I am an atheist.

2. I take no position here on the underlying meaning of IQ testing. I have tried to refer to scores on tests that were adapted from IQ tests and their IQ-equivalent scores (assuming a mean of 100 and a std. dev. of 15) without treating scores here as valid or reliable measures of verbal or general intelligence. I am not, however, in the group who thinks that they measure nothing; there is enough evidence of their correlation with other things that matter (like education, income, and happiness) that it would seem that IQ tests measure something, whatever that might be.

3. GSS data from the 1972-2012 cumulative datafile are weighted by WTSSALL, adjusting for subsampling and the number of adults in the household. I computed vocabulary and analogy scores from the WORD and ALIKE variables, correcting for some slight composition errors in the GSS WORDSUM variable.