"Standard deviations? Linear regressions? Can't we just go outside and play freeze tag?" (Steve Ruark/AP)

A new lawsuit challenging protections for public school teachers was filed Monday, this time in New York. It is modeled on last month's case in which a judge threw out California's laws on tenure on the grounds that they effectively deprived students of an equal right to an education.

The plaintiffs in Wright v. New York, a group of families led by education reformer and erstwhile news anchor Campbell Brown, might not have as strong a case as did the winners of the California case, writes Stephen Sawchuk in Education Week. New York's tenure rules are not as strict as California's. Also, the New York plaintiffs do not claim that children of racial and ethnic minorities are especially likely to have ineffective teachers, which was the issue in the California case. Instead, the group appeals to the right granted to children in New York's state constitution to a basic education.

In other respects, though, the cases are similar. Both cite as evidence a study by celebrity economist Raj Chetty of Harvard University, his colleague John Friedman and Columbia University's Jonah Rockoff. The merits of their work have become the subject of a long-running debate among experts in education policy.

The three economists examined school records, test scores and tax returns for millions of children and young adults over two decades and found that certain teachers' students did better than others in adulthood. The authors concluded that a student who had a better teacher (as defined by test scores) for one year in elementary school was slightly more likely to go to college and would have a slightly higher income at age 28.

The study was covered widely in the media when Chetty, Friedman and Rockoff first started discussing their findings a few years ago, and President Obama used their numbers in his State of the Union in 2012. Now, their  results are no longer just a talking point. The judge in the California case, Rolf Treu, cited them extensively in his opinion, which might encourage legal challenges in states beyond New York and California as well.

Their results have also received criticism -- perhaps unsurprisingly, given how attempts to measure the value added by classroom teachers have aroused suspicion and distrust nationwide among educators, unions and some parents.

One issue is the authors' estimate of slightly greater earnings in adulthood for students who had good teachers in their data. The authors found that compared to an average teacher, a good teacher, defined as the 16th most competent in a ranking of 100 random teachers, could improve the future earnings of students when they are 28 by 1.3 percent.

It's not much, but it's still something, especially since research on teachers and test scores generally shows that even the very best teachers have a difficult time improving their students' scores above the run of the mill. Specifically, if you lined up 1,000 schoolteachers and chose the most competent using Chetty, Friedman and Rockoff's measure, you could expect that teacher's students to rank somewhere in the 61st percentile on tests rather than the 50th percentile, based on the authors' data.

(For stats nerds only: The authors find that for every standard deviation in improvement in the quality of instruction, earnings at age 28 increase by 1.3 percent, and students' test scores improve by one tenth of a standard deviation. That second result confirms what other research in the field has found.)

Yet economist Moshe Adler, also of Columbia, suggests even this modest improvement in earnings could depend on the authors' choice of methods. The authors threw out their result for 30-year-olds because they felt they didn't have a large enough group to draw a firm conclusion. Instead, they assume that the 1.3-percent advantage will persist throughout the worker's career.

Alder argues that the sample was large enough, and the fact that the data yielded no statistically significant conclusions shows that the small advantage in earnings might be short-lived. "They assume that what they found at age 28 holds for a person’s lifetime," he said. "It is dishonest." The paper's authors counter that the advantage of those who have had better teachers seems to increase with age, so the estimate in the paper is conservative. They and Adler have gone back and forth over the past few months, discussing several objections he has to the paper.

Chetty, Friedman and Rockoff found that the slight difference in earnings between those who had better and worse teachers as children seems to increase with age, at least through early adulthood. (National Bureau of Economic Research)

If the percentage is accurate, then in the aggregate, it is a large number. "We know a good teacher can increase the lifetime income of a classroom by over $250,000," Obama said in 2012. Without discounting the cost of inflation and interest, the increase is actually greater than $1 million. In other words, a hypothetical classroom full of parents at Back-to-School Night should be willing to throw their money in a pot and pay $250,000 to get the best teacher in a school with 20 randomly selected teachers.

That hypothetical classroom full of parents doesn't exist, of course, and identifying the best teachers is not a challenge that anyone is quite sure how to solve. Is dismissing teachers whose kids don't do well the right solution, as the groups behind the recent lawsuits argue?

Friedman isn't sure yet how teachers would respond to a system where they faced consequences based on their students' performance. "To me, that’s the really big unknown," he said.

The reason some people have better career prospects when they're 28 depending on who they had as a teacher when they were 10 might not be that they learned how to do long division exceptionally well. Instead, Friedman suggested, maybe they learned something else from their teachers -- perseverance, organization, collaboration -- that helped them on the test and for the rest of their lives. In that case, an exclusive focus on test results could hurt kids.

Still, experimenting with merit pay and limited tenure is worthwhile in Friedman's view, as long as everyone recognizes that teachers alone can't fix a broken public education system. "It's a good thing, but it's not going to solve all the problems of the world," he said.