A study based on new data from the experiment, conducted 14 years ago in about 100 middle-of-the-road Israeli schools, appears to be the first of its kind -- examining the consequences of an alternative model of teacher compensation over the long term. Victor Lavy, the author and an economist at the University of Warwick, found that those students whose teachers were were paid more didn't just score higher on the tests. They went on to complete more years of postsecondary education and to earn more than their peers whose teachers were paid conventionally.
Those results suggest that the teachers provided the students with knowledge or habits that served them well later on life, rather than just, say, teaching them when to guess on a multiple-choice question.
"There is a big concern about whether teachers behave strategically and they only manipulate the test-taking skills of students," Lavy said. "You don't care per se about test scores."
At the end of high school, Israeli students spend a few months doing nothing but preparing for a demanding test called the Bagrut. Their results on the test determine where they can apply for college. Lavy's program offered significant bonuses to teachers if their students showed improvement in their grades on the test. Close to half of the 629 teachers in the experiment received awards, which ranged in value from $1,750 to $7,500.
Offering bonuses to teachers improved their students' scores on the test and increased the number of students attending one of Israel's seven elite research universities by more than a quarter. Lavy found that people who had been involved in the experiment in high school are now earning about 7 percent more than those who didn't, and are 2 percent less likely to claim unemployment benefits.
In interviews after the experiment, teachers told Lavy the bonuses encouraged them to stay after school to help students study or to change their lesson plans, not just to train them in how to take the test.
It shouldn't come as a surprise if teachers work harder when money is on the line, Lavy argued.
"It's very nice to say that being a teacher is a destiny, a mission. That’s all nice," he said. "You really want to motivate people with financial compensation. Teachers are not different from other people."
The study, however, is unlikely to persuade skeptics of pay for performance. Even if the teachers were simply teaching to the test, their students still would have benefited over the long term from being able to attend more selective schools. With slightly better test scores, they had a chance to earn more impressive degrees, so perhaps it's no surprise that these students were earning more a decade later.
If so, then the merit-pay program wouldn't have any effect if implemented nationally in Israel, since the universities would have to respond to higher average test scores by raising the bar for admission. "This could all be a zero-sum game," said Douglas Harris, an economist at Tulane University. "It's coming at the expense of other students."
Meanwhile, there is no equivalent to the Bagrut in the U.S. system, and in contrast to other countries, studies conducted here have not produced convincing evidence on the whole that the programs result in higher test scores even in the short term.
Harris praised Lavy's work as "quite significant," but disagreed with his assessment of teachers' motivations, saying that financial bonuses might not mean much to them. "Most teachers aren't going into it for the money," he said. "They're very much focusing on what they think is in the interest of children."
Lavy also criticized pay-for-performance programs in U.S. schools. In order for them to work, he said, they must be transparent enough for teachers to understand what they have to do to receive a bonus. Teachers receiving ratings based on their students' test scores in places like Houston and Ohio have complained they don't understand why their marks fluctuate from year to year.
"Teachers should know, they should feel with their own hands and fingers, their chances in what they do," Lavy said.
A survey of teachers in Minneapolis found widespread misconceptions about how their merit-pay program was supposed to work. And a recent review of federal grants given to districts for experiments in pay for performance found that half of the teachers in the experiments didn't even know they were eligible for bonuses.
"Teacher pay-for-performance programs are incredibly complex and difficult to implement," Vanderbilt University's Matthew Springer said. "Buy-in from teachers into the program itself is going to be critically important."
Springer and other researchers in the United States are now less interested in competitive merit-pay systems like the one that Lavy designed than they are in simply offering better teachers higher salaries or bonuses to transfer into disadvantaged schools. How those teachers should be identified is still a point of debate, but that kind of system is easier for teachers to understand.
It's the sort of reform that appeals to Kate Walsh, president of the National Council on Teacher Quality, who argues that teachers' pay shouldn't fluctuate with their children's scores from year to year.
"The real issue for the profession is you need to pay highly effective teachers higher salaries," she said, citing six figures. "What's wrong with the profession is that talented people look at it and say, 'I'm not going to do this because no matter how long I do this, I'm going to get paid just as much people who aren't that good, and not only that, but I don't know if I'll ever be able to buy a house on this salary."
For his part, Lavy does not advocate for any particular model of pay for performance, or one based exclusively on test scores. After all, placing so much important on a single test like the Bagrut can be unfair to students, since a single bad day can damage their chances of getting into the college of their choice and pursuing their academic interests. In another recent study, Lavy and two collaborators showed that taking a test on a day with bad air pollution reduced students' earnings fifteen years later by about 2 percent.
Rather, Lavy says his results show the benefits of offering administrators and policymakers more flexibility in compensating teachers, even if they don't rely on noisy test scores to make their judgments.
"Paying individuals based on their productivity is what drives the American economy," he said. "Why not for teachers?"