wpostServer: http://css.washingtonpost.com/wpost

The Post Most: Local

Answer Sheet
Posted at 09:01 PM ET, 01/25/2012

Readability formulas and Obama’s speech, Gingrich’s PhD

Folks at the University of Minnesota’s Smart Politics blog ran President Obama’s 2012 State of the Union address through a “readability formula” and reported that it scored as having been written at an 8.4 grade level.

That is better than the first act of the absurdist Samuel Beckett play “Waiting for Godot” scored on the same formula; it earned a 4.08 grade level, though you aren’t likely to find the play in many fourth-grade classes. Obama’s score is only slightly higher than the 8.93 grade level earned by Newt Gingrich’s doctoral dissertation (titled “Belgian Education Policy in the Congo, 1945-1960), also on the same formula.

Smart Politics has run 70 Sate of the Union addresses delivered since 1934 through what is known as the Flesch-Kincaid readability test, one of several dozen formulas that purport to detect the difficulty of a particular text, and, therefore, at what grade level it is written.

The formulas essentially measure a work’s average sentence length, average word length in number of letters, word difficulty level, and total number of words. But “readability formulas cannot measure context, prior knowledge, interest level, difficulty of concepts,” according to a June 2011 report by Renaissance Learning Inc.about these formulas.

The report says they can’t even measure whether a work is coherent. According to the report, the first two lines of Lincoln’s Gettysburg address would be scored exactly the same as the same two lines when written backwards.

“Four score and seven years ago our fathers brought forth upon this continent a new nation, conceived in liberty and dedicated to the proposition that all men are created equal. Now we are engaged in a great civil war, testing whether that nation or any nation so conceived and so dedicated can long endure.”

Those lines would have the same score as these:

“Endure long can dedicated so and conceived so nation any or nation that whether testing, war civil great a in engaged are we now. Equal created are men all that proposition the to dedicated and liberty in conceived, nation new a continent this upon forth brought fathers our ago years seven and score four.”

Renaissance Learning owns a major reading program called Accelerated Reader that lets students pick books to read and then gives them points based on a readability formula that. The program is in 15,000 schools across the United States and more than 2,500 schools in Great Britain.

The formula used to give points to books in the Accelerated Reader program gives 28 points to the last book in the “Twilight” series by Stephenie Meyer, “Breaking Dawn,” about a girl who becomes a vampire, and 25 points to Steinback’s classic “The Grapes of Wrath.” And Dan Brown’s “The Da Vinci Code” gets more points, 23, than Harper Lee’s classic “To Kill a Mockingbird,”15.

Gingrich’s PhD got lower scores on other formulas. For example, on something called the Automated Readability Index, the score was 7.48 grade level, though others ranked it at early 11th grade.

Since we know the formulas don’t tell us much, we have to actually read the pieces to know how well they have been written. Gingrich’s PhD introduction starts like this (and no, this is not a middle school geography report):

“The Congo is a large country, about the size of the United States east of the Mississippi. It is located in what then United States Assistant Secretary of State for African Affairs G. Mennen Williams termed “the geographic and strategic heart of Africa.” It has the third largest gross national product in Africa south of the Sahara. The country’s physical and economic characterizations are so impressive that, in George Ball’s words, “What happens in this former Belgian colony will obviously play a decisive role in what happens in the areas around it.”

So what have we learned from all of this?

Perhaps that the University of Minnesota’s annual State of the Union analysis on its Smart Politics blog isn’t quite as smart as it should be.

-0-

Follow The Answer Sheet every day by bookmarking http://www.washingtonpost.com/blogs/answer-sheet.

By  |  09:01 PM ET, 01/25/2012

 
Read what others are saying
     

    © 2011 The Washington Post Company