The Washington Post

The denominator fallacy rears its ugly head


This fallacy is so well known that you don’t see it much anymore, when people make comparisons without correcting for the obvious denominator.

For example, graphs that show the dramatic rise over time in the cost on some item or another in nominal dollars, not adjusting for inflation. Or comparisons between states or countries that don’t adjust for population (e.g., look how many more murders occur in New Jersey than in Mississippi!).

Anyway, as I said, this particular error has mostly gone out of style, but statistical connoisseurs enjoy spotting it on the rare occasions that we notice it.

In this case, I was browsing the Instapundit blog of Glenn Reynolds and noticed the following item:

THE MYTH OF “RED STATE WELFARE:” In fact, since 2000, solid “blue” states have received far more in federal funding than solid “red” states. “Here is a breakdown of all 50 states and the District of Columbia by electoral map color and federal funding since 2000, according to USA.gov.”

I saw this and was, like, huh? Really? I followed the link to this October 2012 news article at The Blaze by Becket Adams, who wrote:

Since 2000, solid “blue” states have received far more in federal funding than solid “red” states, a Blaze analysis finds.

But first, let’s define the what we mean by “red” state, “blue” state, “purple” state, etc.:

Red: Republican presidential candidates have won this state in every election since 1996
Pink: Republican presidential candidates have won this state three times since 1996
Purple: This state has been won twice by a Republican presidential candidate and twice by a Democrat presidential candidate since 1996
Light Blue: Democrat presidential candidates have won this state three times since 1996
Blue: Democrat presidential candidates have won this state in every election since 1996 . . .

And then a graph, showing that “Strong Dem” states got $12.2 trillion, “Likely Dem” states got $0.6 trillion, “50/50″ states got $4.8 trillion, “Likely GOP” states got $5.8 trillion, and “Strong GOP” states got $5.6 trillion.

OK, the obvious thought: from 1996-2008, the “Strong Dem” states include a lot more people than the “Strong GOP” states. I did a quick count and got the following totals (based on 2000 population), again using Becket Adams’s definitions:

“Strong Dem”: 136 million people
“Likely Dem”: 6 million people
“50/50″: 29 million people
“Likely GOP”: 55 million people
“Strong GOP”: 55 million people

And, of course, if you divide the Adams’s spending totals by the population, you get much more reasonable ratios:

“Strong Dem”: $90,000/person
“Likely Dem”: $97,000/person
“50/50″: $162,000/person
“Likely GOP”: $105,000/person
“Strong GOP”: $101,000/person

I purposely didn’t make a graph of these numbers, because I didn’t look into Adams’s numbers in detail, and also I classified the states quickly and might have missed one or two, and beyond all that, the above partition of states is a silly and arbitrary way to categorize them. And I have no interest in making any political points here. It’s already well known that Republican-leaning states tend to get more money from the government than they pay in taxes, and I don’t have anything to add to this discussion by presenting these particular aggregates.

No, my reason for sharing this story is that, nowadays, the denominator issue is so well known that it’s a delightful rarity to see the mistake in print (or in pixels, as the case may be).

The natural question is: did Adams know he was presenting distorted statistics, or was he making an honest mistake? I have no idea. On one hand, the mistake is so clear and embarrassing that no self-respecting political writer would want to make it in public. On the other hand, it was the month before the presidential election and Adams might well have felt the pressure to come up with some statistics—any statistics—to make Democrats look bad, and maybe that’s the best he could do.

A bad sign is that some of his commenters noted the mistake but he did not apologize or even silently correct the error. (He did write, “Moreover, a state’s population plays a large role in the total amount received,” but that seems more like he was trying to dodge or swat away the criticism rather than be serious.) And did Reynolds realize he was passing along junk to his audience? Again, I have no idea, but I’m guessing that he posts so much on his blog every day that, like David Brooks, he has no time to read the comments and correct his errors.

To be sure, it’s an inconsequential mistake from over a year ago that just happened to reappear on a popular aggregator.  But it’s just so rare these days that people forget the denominator that it’s fun to spot it when it happens.

Andrew Gelman is a professor of statistics and political science at Columbia University. His books include Bayesian Data Analysis; Teaching Statistics: A Bag of Tricks; and Red State, Blue State, Rich State, Poor State: Why Americans Vote the Way They Do.

The Freddie Gray case

Please provide a valid email address.

You’re all set!

Campaign 2016 Email Updates

Please provide a valid email address.

You’re all set!
Comments
Show Comments
The Republicans debate Saturday night. The New Hampshire primary is on Feb. 9. Get caught up on the race.
Heading into the next debate...
Donald Trump returns to the Republican presidential debate stage Saturday night. Marco Rubio arrives as a sudden star, but fending off ferocious attacks from his rivals. Still glowing from his Iowa victory, Ted Cruz is trying to consolidate conservative support, while Ben Carson is struggling to avoid being typecast as the dead man walking.
Listen
Play Video
New Hampshire polling averages
Donald Trump holds a commanding lead in the next state to vote.
New Hampshire polling averages
Polling in New Hampshire has typically been volatile after Iowa's caucuses, but Bernie Sanders, from its neighboring state Vermont, has been holding a lead over Hillary Clinton.
55% 38%
Listen
Play Video
Upcoming debates
Feb. 6: GOP debate

on ABC News, in Manchester, N.H.

Feb. 11: Democratic debate

on PBS, in Wisconsin

Feb 13: GOP debate

on CBS News, in South Carolina

Campaign 2016
State of the race

To keep reading, please enter your email address.

You’ll also receive from The Washington Post:
  • A free 6-week digital subscription
  • Our daily newsletter in your inbox

Please enter a valid email address

I have read and agree to the Terms of Service and Privacy Policy.

Please indicate agreement.

Thank you.

Check your inbox. We’ve sent an email explaining how to set up an account and activate your free digital subscription.