I can't remember helping inspire a research report before, so I was very pleased to receive recently from the Center for Civic Innovation at the Manhattan Institute for Policy Research a working paper that began as an e-mail argument, waged through moi, between two of America's most provocative educational researchers.
This happened several months ago. I can't be expected at my age to remember exactly what triggered the argument between Manhattan Institute senior fellow Jay P. Greene and George Mason University educational psychologist Gerald W. Bracey. It was something about a column | http://www.washingtonpost.com/ac2/wp-dyn?pagename=article&contentId=A52280-2004Jan27¬Found=true I did on Bracey's views of Simpson's Paradox, a statistical quirk that we journalists often trip over when reporting test scores of different ethnic groups. The e-mails started coming at me pretty fast. I sent Greene's to Bracey and Bracey's to Greene, and watched the word temperature rise.
It was an interesting dispute, about the meager gains in achievement in the past 30 years and their relation to changes in the lives of low-income children. I wasn't able to contribute much, but I enjoyed the discussion, as I do all arguments between smart people on important topics.
Now Greene and Manhattan Institute senior research associate Greg Forster have written, partially in response to what Bracey was saying that day, a paper | http://www.manhattan-institute.org/html/ewp_06.htm that I am sure will stimulate more feverish debate. It is called "The Teachability Index: Can Disadvantaged Students Learn?"
Greene and Forster, very ambitious men, have pushed the issue far beyond the point I would have imagined was possible. Not only do they conclude that low-income children's lives have improved a bit in the past three decades, but they also suggest a new way of measuring the disadvantages of inner city and rural lives, and they rank every state in the country, plus the District of Columbia, on a scale that attempts to show who is doing the best job teaching those kids.
I have never seen anything quite like it. I advise you to read it and judge for yourself. It delves into many statistical technicalities, but in clear prose that even I could follow. It is a consumer's guide to the tools social scientists might use to measure the pain, stress and confusion of these young lives, and the pitfalls of doing so.
Bracey says "the paper presents what are, at best, hypotheses as if they were facts and uses insinuations and out of context quotes to direct the reader to its desired conclusions." Richard Rothstein, a national expert on school spending who is criticized in the report, said Greene's and Forster's assumptions on school cost increases are wrong and their measurements based on arbitrary calculations.
Bracey and Rothstein, research associate of the Economic Policy Institute and visiting professor at Teachers College Columbia University, will have more to say at the end of this column, but I think the report is still worth reading because it dissects in an intriguing way an issue that is at the root of every book, conference, course and article on how to improve our schools: can educators really be expected to teach very much to kids who have so many burdens and distractions?
Greene and Forster put it this way: "Reformers argue that the school system needs systematic reform because it is not performing as well as it should in light of the enormous increases in resources that have been poured into it: inflation-adjusted education spending per pupil has doubled in the past 30 years, while student achievement and graduation rates have remained flat. Defenders of the public school system often respond by pointing to the difficulty of educating students who face disadvantages that make them less teachable, arguing that these challenges consume the system's resources and hinder its performance."
That was Bracey's and Rothstein's view, more or less. The new paper is Greene's way of saying, "Oh, yeah?"
He and Forster have broken down the lives of the young and poor into six separate indexes, which taken together form what they call the Teachability Index. The six component indexes are:
* Readiness (including data on preschool enrollment, language other than English, and parents' education).
* Economics (family income and poverty levels).
* Community (crime victimization, drug use, religious observance and residential mobility).
* Health (disabilities, mortality, low birth-weight survival and suicide).
* Race (the portion of non-Hispanic whites among school children).
* Family (teenage birth and single parenthood).
Greene and Forster say they weighted each of their six indexes equally, having no scientific data that would tell them which problems affect children more than others.
They pick several fights along the way. For instance, they accuse David Berliner and Bruce Biddle, whose book "The Manufactured Crisis" is a bible for those defending the work of public schools, of providing no evidence for their argument that child health needs have increased and of inflating the level of poverty and extent of disabilities among American children.
You have to read the paper to get the full flavor of the argument. Once Berliner, Biddle and the paper's other targets write their own responses, I will point you to them. But I will summarize Greene's and Forster's complaints about disability statistics because the issue is extremely important in understanding how schools work these days.
The two researchers say they have not counted students designated with specific learning disabilities because they think large numbers of children get those labels only because their school districts want more special education dollars. They note that Berliner and Biddle put great emphasis on the growth in the portion of special education students from 8.3 percent in 1976-77 to 13.3 percent in 2000-1. They say that does not fit with other changes over the same period, such as a drop in the number of students classified as mentally retarded from 961,000 to 599,000 in the same period.
At the end they admit that "the Teachability Index is not precise enough to allow for detailed statistical analysis" of their original question, why more money for schools did not produce more achievement, but it does, they say, cast great doubt on the notion that kids' lives got worse and made them harder to teach. "The evidence indicates that on the whole, students are easier to teach today than they have been at any time in the past 30 years," they say. They also argue that the results show that some states have found a way to reach low-income children, even if the overall national picture is pretty dismal.
Bracey, in an e-mail to me, said Greene was unethical to release this paper, which has not been reviewed by other researchers, shortly after he signed an advertisement in the New York Times criticizing coverage of a study of charter schools that had similarly not been peer-reviewed.
Greene said he has sent the paper around for review by others, and was instead warning journalists in the ad to make sure they have critical voices in stories about such studies. That is a good idea, so here is more of what may be wrong with The Teachability Index:
Rothstein's research has shown that much increased spending in schools has helped children with disabilities or those who are unfamiliar with English. "Some of the real increase in spending over the last 30 years," he says, "has gone for the education of children who were not handled by the public school system 30 years ago. So if you calculate real spending increases with a reasonable price adjustor [Rothstein says Greene's and Forster's inflation rate is flawed], and then further remove spending increases that should not properly be expected to improve test scores, you have a much, much smaller increase in real spending over 30 years -- less than half the doubling Greene claims."
He said Greene's conclusions about teachability rely on an arbitrary system not only for choosing indicators but for weighting them. "A different and equally arbitrary weighting system could lead to entirely different conclusions about teachability," he said. "For example, low birth-weight survival is on-fourth of the health composite, but school racial integration is 100 percent of race composite. Thus, in his final index, low birth-weight survival is one-fourth as important as race (and half as important as poverty or single parenthood.)"
Bracey says "the paper . . . is easily dismissed: It does not deal with the plague of asthma among the urban poor (and the resulting school attendance problems), nor the legion of stories indicating that violence, behavioral and emotional disorders, and disrespect for teachers are much more prevalent than in the past, even among the youngest students. A recent survey in Tarrant County (Fort Worth) Texas found that 86 percent of elementary school counselors said that the time of assistant principals was being diverted to behavior management problems. Eight-five percent said today's kindergartners have more emotional or behavioral problems than they did a mere five years ago.
"There are many stories today about how children are 'disconnected' from their schools," Bracey said. "Disconnected kids are not teachable. And, according to African-American analysts such as John McWhorter, the anti-intellectualism of hip-hop cultures has retarded achievement in the black community generally. No one, certainly not Bill Cosby, has ever accused hip-hop of making kids more teachable in school."
"Anyone who has spent any time in classrooms knows that the classroom learning atmosphere does not necessarily reflect the behavior of the average child. One or two disruptive kids can create an unteachable environment. The paper deals not at all with classroom environment (nor, oddly enough, teacher qualifications that also vary with the poverty level of the school and no doubt interact with any 'teachability' of the kids.)"
So maybe this is not the right formula, but it would be nice if we could find one that was. Teachability is important, and measuring it accurately might help many kids.