The critique, which was published on the academic platform PeerJ in August, alleges that researchers have exaggerated the benefits of a program that is now used as a model for healthy eating in schools. The critique is preliminary and has not been peer-reviewed.
The paper raises questions about the efficacy of the Smarter Lunchrooms Movement, which has been adopted by more than 30,000 schools across the U.S. since its launch in 2010. And it speaks volumes about the challenges of trying to get children (and adults) to voluntarily improve their diets.
In addition to the Smarter Lunchrooms Movement, USDA has devoted millions of dollars to improving the nutrition of school meals and encouraging food-stamp recipients to buy more produce.
Both efforts produced results on par with a single bite of apple, said David Just, the co-director of Cornell University's Center for Behavioral Economics in Child Nutrition Programs, the research group that administers the Smarter Lunchrooms Movement.
Any nutrition intervention — including Smarter Lunchrooms — comes with striking limitations, Just acknowledges.
“The best we would hope for, under ideal conditions, is to get kids to eat some extra fraction of a serving of fruits or vegetables,” Just said.
Since 2010, the USDA-funded Smarter Lunchrooms program has often been held up as a model for persuading children to make healthier choices.
Using the principles of behavioral economics, a field that studies why people make decisions, Smarter Lunchrooms encourages school food service workers to make cosmetic changes that “nudge” students toward healthier choices. Administrators may place fruits in attractive baskets, for instance, or assign catchy names to vegetable dishes.
Schools that adopt these techniques can earn awards of up to $2,000 from USDA. More than $5.5 million in grants have also been earmarked for Smarter Lunchrooms training.
But the research methods employed by the Smarter Lunchrooms team have fallen under scrutiny, forcing Cornell to open an investigation into its practices earlier this year. Six months ago, after one of the team's directors published a blog post that described unorthodox statistical methods, a team of three independent researchers raised concerns about a number of the lab’s papers.
Andrew Gelman, a statistician at Columbia University, accused the lab of using statistical manipulations — what he called “junk science” — designed to make its findings look more significant.
More recently, Eric Robinson, a behavioral scientist at the University of Liverpool who researches eating behavior and obesity, reviewed a series of BEN Center papers that informed the Smarter Lunchrooms techniques. He found that several appeared to exaggerate the available data or over-generalize their findings.
In one instance, a series of interventions that were said to “significantly” increase children’s fruit consumption only led them to eat the equivalent of an extra one-tenth of a small apple per day — the aforementioned apple bite.
In another paper, the Cornell researchers concluded that giving vegetable dishes fun names — a core recommendation of the program — “persistently increased” vegetable consumption. In reality, however, the study only looked at how many vegetables kids put on their plates, not how many they actually ate, over an extended time period.
In a statement, a spokeswoman for USDA said the agency had raised some concerns with the BEN Center and that they were being reviewed. But there is no plan to remove Smarter Lunchrooms from schools.
“It’s important to remember that Smarter Lunchrooms strategies are based upon widely researched principles of behavioral economics, as well as a strong body of practice that supports their ongoing use,” the spokeswoman said.
Cornell’s BEN Center has also defended its work. In a prepared statement, the lab said its Smarter Lunchrooms techniques were based on “a wide body of behavioral literature” beyond the studies cited by Robinson. It also disputed a number of his observations, accusing him of taking certain phrases or statistics out of context.
Just, who described Robinson as a “solid researcher,” believes much of the hullabaloo boils down to perspective: namely, whether it is fair to describe a small change, such as a bite of apple, as “significant.”
This is a more nuanced question than it may seem: It is notoriously difficult to change people’s eating habits, which become ingrained during early childhood. According to the Centers for Disease Control and Prevention, 87 percent of Americans don’t eat enough vegetables, and 76 percent don’t eat enough fruit. (Those figures are 93 percent and 60 percent, respectively, for children.) In that context, Just said, even an extra fraction of a serving of fruits or vegetables is good.
At the same time, Robinson points out, such a small uptick is unlikely to have any real impact on an individual’s health. There is little evidence that such nudges build on each other, causing children to adopt better long-term habits. And such a small increase still leaves most children eating less produce than is recommended to lower their risk of illnesses such as heart disease and diabetes.
“[The nudge] is a popular idea at the moment and it sounds reasonable,” Robinson said. “The problem with this idea is that it is only likely to have a very small effect on nutrition.”
Such debates could become more important than ever in the coming months.
In 2018, Congress will vote to reauthorize the Farm Bill — the legislation that funds the Supplemental Nutrition Assistance Program, formerly known as food stamps. Ahead of that vote, some public health groups and policymakers have begun to make the case for new nutrition interventions in the program, potentially including some type of “nudges.”
In a February congressional hearing, Brian Wansink — the co-director, with Just, of Cornell’s BEN Center — was asked how nudge techniques like those in Smarter Lunchrooms could be extrapolated to grocery stores. Wansink suggested the government train retailers on those strategies, such as placing fruit by the cash register, and give incentives to stores that adopt them.
But that gives some researchers pause, especially in light of Robinson's analysis. Craig Gundersen, an agricultural economist at the University of Illinois, said he is particularly concerned that the Smarter Lunchrooms approach is distracting policymakers from more effective — and less politically palatable — options, such as increasing funding for the school meals program.
“Shouldn't we be doing that instead of moving around the apples and bananas and broccoli?” Gundersen asked. “I haven't seen any evidence that these things really work, in or out of the school context.”