Yes, numbers (on international tests) can lie

What must be the best analysis so far of the newly released results of  international reading, math and science exams comes from the University of Oregon’s Yong Zhao, who tells us that numbers can, in fact, lie. Referring to the scores on the 2011 Progress in International Reading Literacy Study, known as PIRLS, and the Trends in International Mathematics and Science Study, referred to as TIMSS, he tells us:

“Numbers don’t lie,” many may say but what truth do they tell? Look at the following numbers:

Table 1: Scores and Attitudes of 8th Graders in TIMSS 2011

Country Math Scores Confidence (%) (4th Grade) Value Math (%)
Korea 613 03 (11) 14
Singapore 611 14 (21) 43
Chinese Taipei 609 07 (20) 13
Hong Kong 586 07 (24) 26
Japan 570 02 (09) 13
United States 509 24 (40) 51
England 507 16 (33) 48
Australia 505 17 (38) 46

These are the scores of 8th graders and percentage of them saying they are confident in math and value math. Top scoring Korea has only 3% of students feeling confident in their math and 14% valuing math, in contrast is Australia with much lower scores but significantly higher percentage of students feeling confident in math and valuing math. In fact, the top 5 East Asian countries in math scores have way fewer students reporting confidence in math and valuing math than the U.S., England, and Australia, all scored significantly lower.

It gives me a headache to understand these numbers: Do they mean that even if the Korean students do not think math is important, they study it anyway? and they have a very effective education that can make people who do not value math to be outstanding in it? Or since these are 8th graders, do they mean that after learning math for 8 years, the students feel the math they have been learning is not important in life? In the case of the United States, do they mean that American students value math but have poor math learning experiences that lead to low math achievement? Or could it be that their 8 years of math learning convinced them, at least a much larger proportion than in Korea, that math is important?

The same questions can be asked about confidence. Do the numbers mean that Korean students lack of confidence makes them study harder so they achieve better in math than their American or Australian counterparts? Or could they mean that the way math is taught in Korea made them lose confidence in math?

The data show that as students progress toward higher grades, they become less confident in their math learning. More fourth graders than eighth graders have confidence in math, for example.  Does this mean the more they learn, the less confident they become?

Or perhaps these numbers are not related at all. But the TIMSS report suggests that within countries students with higher scores are more likely to have a more positive attitude towards math, that is, a positive correlation. A negative correlation is found between countries and has been a pattern as Tom Loveless discovered in previous TIMSS. So somehow math scores, attitudes, and confidence are related. Perhaps whatever in an education system or culture that boosts math scores leads to less positive attitude and lower confidence at the same time. In this case, one needs to ask what is more important: scores, or confidence and positive attitude?

There can be other interpretations but whatever the interpretation is, these numbers show that results of TIMSS, or other international assessments such as the PISA, are a lot more complex than what the headlines attempt to suggest: Asians are great, America sucks, so do Australia and England. The TIMSS and PISA scores are perhaps worth much less than politicians and the media make of them, as the rest of this paper shows.

After showing how the United States has had low international test scores for more than 50 years, he writes:

So far all international test scores measure the extent to which an education system effectively transmits prescribed content.

In this regard, the U.S. education system is a failure and has been one for a long time.

But the successful transmission of prescribed content contributes little to economies that require creative and entrepreneurial individual talents and in fact can damage the creative and entrepreneurial spirit. Thus high test scores of a nation can come at the cost of entrepreneurial and creative capacity.

While the U.S. has failed to produce homogenous, compliant, and standardized employees, it has preserved a certain level of creativity and entrepreneurship. In other words, while the U.S. is still pursuing an employee-oriented education model, it is much less successful in stifling creativity and suppressing entrepreneurship.

The U.S. success in creativity and entrepreneurship is merely an accidental by product of a less successful employee-oriented education, which is far from sufficient to meet the coming challenges brought about by globalization and technological changes. Thus in a sense, the U.S. education is in turmoil, inadequate, and obsolete, but it has to move toward more entrepreneur-oriented instead of more employee-oriented.

Read the whole post here.

Yong Zhao is the presidential chair and associate dean for global education at the University of Oregon’s College of Education, where he also serves as the director of the Center for Advanced Technology in Education.

Valerie Strauss covers education and runs The Answer Sheet blog.
SECTION: {section=local, subsection=null}!!!
INITIAL commentConfig: {includereply=true, canvas_permalink_id=washpost.com/8bvh5zpd9k, allow_comments=true, commentmaxlength=2000, includeshare=true, display_comments=true, canvas_permalink_app_instance=m6yzjj840m, display_more=true, moderationrequired=false, includefeaturenotification=true, defaultsort=reverseChronological, canvas_allcomments_id=washpost.com/km4ey0dajm, comments_period=14, includevoteofftopic=false, allow_videos=false, childrenitemsperpage=3, markerdisplay=post_commenter:Post Commenter|staff:Post Writer|top_commenter:Post Forum|top_local:Washingtologist|top_sports:SuperFan|fact_checker:Fact Checker|post_recommended:Post Recommended|world_watcher:World Watcher|cultuer_connoisseur:Culture Connoisseur|weather_watcher:Capital Weather Watcher|post_contributor:Post Contributor, includesorts=true, includeheader=true, defaulttab=all, includeverifiedcommenters=true, includerecommend=true, maxitemstop=2, includereport=true, source=washpost.com, allow_photos=false, maxitems=7, display_ugc_photos=false, includepause=true, canvas_allcomments_app_instance=6634zxcgfd, includepermalink=false}!!!

UGC FROM ARTICLE: !!!

FINAL commentConfig: {includereply=true, canvas_permalink_id=washpost.com/8bvh5zpd9k, allow_comments=true, commentmaxlength=2000, includeshare=true, display_comments=true, canvas_permalink_app_instance=m6yzjj840m, display_more=true, moderationrequired=false, includefeaturenotification=true, defaultsort=reverseChronological, canvas_allcomments_id=washpost.com/km4ey0dajm, comments_period=14, includevoteofftopic=false, allow_videos=false, childrenitemsperpage=3, markerdisplay=post_commenter:Post Commenter|staff:Post Writer|top_commenter:Post Forum|top_local:Washingtologist|top_sports:SuperFan|fact_checker:Fact Checker|post_recommended:Post Recommended|world_watcher:World Watcher|cultuer_connoisseur:Culture Connoisseur|weather_watcher:Capital Weather Watcher|post_contributor:Post Contributor, includesorts=true, includeheader=true, defaulttab=all, includeverifiedcommenters=true, includerecommend=true, maxitemstop=2, includereport=true, source=washpost.com, allow_photos=false, maxitems=7, display_ugc_photos=false, includepause=true, canvas_allcomments_app_instance=6634zxcgfd, includepermalink=false}!!
Comments
SECTION: {section=local, subsection=null}!!!
INITIAL commentConfig: {includereply=true, canvas_permalink_id=washpost.com/8bvh5zpd9k, allow_comments=true, commentmaxlength=2000, includeshare=true, display_comments=true, canvas_permalink_app_instance=m6yzjj840m, display_more=true, moderationrequired=false, includefeaturenotification=true, defaultsort=reverseChronological, canvas_allcomments_id=washpost.com/km4ey0dajm, comments_period=14, includevoteofftopic=false, allow_videos=false, childrenitemsperpage=3, markerdisplay=post_commenter:Post Commenter|staff:Post Writer|top_commenter:Post Forum|top_local:Washingtologist|top_sports:SuperFan|fact_checker:Fact Checker|post_recommended:Post Recommended|world_watcher:World Watcher|cultuer_connoisseur:Culture Connoisseur|weather_watcher:Capital Weather Watcher|post_contributor:Post Contributor, includesorts=true, includeheader=true, defaulttab=all, includeverifiedcommenters=true, includerecommend=true, maxitemstop=2, includereport=true, source=washpost.com, allow_photos=false, maxitems=7, display_ugc_photos=false, includepause=true, canvas_allcomments_app_instance=6634zxcgfd, includepermalink=false}!!!

UGC FROM ARTICLE: !!!

FINAL commentConfig: {includereply=true, canvas_permalink_id=washpost.com/8bvh5zpd9k, allow_comments=true, commentmaxlength=2000, includeshare=true, display_comments=true, canvas_permalink_app_instance=m6yzjj840m, display_more=true, moderationrequired=false, includefeaturenotification=true, defaultsort=reverseChronological, canvas_allcomments_id=washpost.com/km4ey0dajm, comments_period=14, includevoteofftopic=false, allow_videos=false, childrenitemsperpage=3, markerdisplay=post_commenter:Post Commenter|staff:Post Writer|top_commenter:Post Forum|top_local:Washingtologist|top_sports:SuperFan|fact_checker:Fact Checker|post_recommended:Post Recommended|world_watcher:World Watcher|cultuer_connoisseur:Culture Connoisseur|weather_watcher:Capital Weather Watcher|post_contributor:Post Contributor, includesorts=true, includeheader=true, defaulttab=all, includeverifiedcommenters=true, includerecommend=true, maxitemstop=2, includereport=true, source=washpost.com, allow_photos=false, maxitems=7, display_ugc_photos=false, includepause=true, canvas_allcomments_app_instance=6634zxcgfd, includepermalink=false}!!
Show Comments
Most Read Local
Next Story
Valerie Strauss · December 12, 2012