Jay Mathews

 TOOLBOX Resize
By Jay Mathews
Monday, October 19, 2009

Politicians and pundits are using results from the Programme for International Student Assessment (PISA) tests to say our kids are falling behind the rest of the world, so maybe we should get some PISA practice. Brookings Institution scholar Tom Loveless, a member of the U.S. advisory board to PISA, offered this sample question for 15-year-olds from the mathematics literacy section of the exam:

For a rock concert a rectangular field of size 100m by 50m was reserved for the audience. The concert was completely sold out and the field was full with all the fans standing. Which one of the following is likely to be the best estimate of the total number of people attending the concert?

A. 2000

B. 5000

C. 20000

D. 50000

E. 100000

I think this is a bad question and not just because I got it wrong. I said 5,000. The answer is 20,000. I don't see why deciding four people, not one, would fit better in a square meter is a sign of math literacy. There are some people I don't like to get close to at concerts.

Loveless, an expert on international testing, agrees that the problem was ill-chosen. "I think it would throw kids off," he said. "The math is rather trivial." There are other problems with PISA, such as an ideological bias and a tendency to assume cause-and-effect relationships. But the U.S. results on those exams -- below average in both math and science literacy -- are often cited as a national disgrace.

A key failing of PISA, Loveless said, is that "it does not measure what kids have learned in school." Why not? Because PISA exams are written by the losing side in a century-old debate over how to teach math. For convenience, call the pro-PISA people progressives and the anti-PISA people traditionalists. Loveless is a traditionalist but appreciates the arguments on both sides. The progressives want to make math instruction more relevant to the real world and emphasize mathematical reasoning more than calculation. The traditionalists say you can't reason well without mastering the fundamentals. They dislike their approach being dismissed as "shopkeeper math," Loveless said, "like it was old-fashioned to try to compute anything."

Loveless prefers the other major international test, Trends in International Mathematics and Science Study, which aligns more closely with the way U.S. students are taught. He is the U.S. representative to the General Assembly of the International Association for the Evaluation of Educational Assessment, a 60-nation body that oversees TIMSS.

Policymakers are right to worry about our world standing in math and reading. But their interpretations of the data are often flawed and make our schools look worse than they are. The Wall Street Journal headlined a 2004 PISA report: "U.S. Teens Are Among Worst at Math." But on TIMSS, we are above the international average in fourth- and eighth-grade math.

Conservative politicians who decry how bad the test makes us look should be careful, as the attitudinal parts of PISA lean left. On PISA's student questionnaire, those who support statements such as "I am in favor of having laws that regulate factory emissions even if this would increase the price of products" are deemed to be environmentally responsible. Students who disagree are not.

This is not to say international comparisons are useless. We can learn from differences between us and the test-takers at the top of the TIMSS (and in some cases the PISA) list: Hong Kong, Korea, Singapore, Taiwan and Japan. Their families devote more out-of-school time to learning. The curricula and textbooks are more focused.

In those places, hardworking students are admired by their peers. Here, they are often dismissed as freakish nerds. "There is a certain anti-intellectualism in our history," Loveless said.

That's okay with me. I think our economy, our politics and our society have benefited from having as much respect for street smarts as book smarts. Being unimpressed with academic degrees saves us from the stultifying aspects of European-style class divisions. But we pay a price. Perhaps it would help if we found ways to celebrate our grade-school math whizzes as much as our playground stars. But more hurrahs, even from 15-year-olds, for people who do their homework, can't hurt.

For more sample questions, go to http://washingtonpost.com/class-struggle.

© 2009 The Washington Post Company