This post has been updated to mention earlier merit pay studies.
There's very good evidence that teacher quality matters a lot in terms of student performance in school and success later on in life. The economist Raj Chetty of Harvard, for example, has found that students randomly placed with more experienced kindergarten teachers not only perform better on tests but earn more and save more for retirement as adults, are likelier to go to college, and go to better colleges than their peers with less experienced teachers. Eric Hanushek of Stanford estimates that a good teacher - defined as at the 84th percentile, or one standard deviation above the mean for you stats nerds - provides students with test scores associated with an increase of between $22,000 and $46,000 in lifetime earnings.
Findings like these lead some to favor "merit pay" regimes that include student test scores as a determinant of teachers' salaries. This has met opposition from teachers' unions and testing skeptics, who argue that it would result in teaching-to-the-test at the expense of actual learning. For a long time, the data has been mixed on merit pay. Two studies from Mathematica Policy Research in 2010 that found little benefit, while a study in Nashville found mild benefits for fifth graders but none for other students.
That has changed with the publication of a new paper (pdf) by Harvard's Roland Fryer, the University of Chicago's Steven Levitt (of Freakonomics fame) and John List, and UC San Diego's Sally Sadoff. The authors went into nine K-8 schools in Chicago Heights, a city 30 miles south of Chicago, and randomly selected teachers (who had to consent, which 93.75 percent did) to take part in a merit pay scheme. The students affected were overwhelmingly low-income, with 98 percent receiving free or subsidized lunches. Teachers in the experiment were offered $80 per percentile improvement in student test scores, for a maximum reward of $8,000, compared to a typical teacher salary of $50,000.
The authors split teachers in the study into a control group, who were not offered any rewards, a "gain" group, which was promised rewards of up to $8,000 at the end of the school year, and a "loss" group, which was given $4,000 upfront and asked to pay back any rewards they did not earn. The idea behind the latter group was that loss aversion should motivate teachers to perform better than they would if they only stood to gain more money. Additionally, the gain and loss groups were split, with a "team" group being rewarded on the basis of theirs and fellow teachers' test scores, and the "individual" group being reward only on the basis of their own scores. The conclusion: it worked, and it worked almost twice as well when the money was given at the start and then taken away:
To get some idea of what the numbers on that graph might mean, using Hanushek's estimates, the testing gains of those of the "loss" group are associated with an increase in lifetime earnings of between $37,180 and $77,740, and those of the "gain" group with an increase of between $20,350 and $42,550. Interestingly enough, it didn't seem to matter much whether the pay was tied to the performance of a given teacher or to the team to which that teacher was assigned. This suggests that a merit pay regime need not pit teachers in a given school against each other to get results.