And that’s why we test

Correction: An earlier version of this article included an incorrect estimate of the number of recipes published each week by the San Francisco Chronicle. It publishes seven to 10 recipes per week, not 15 to 20. This version has been corrected.


The Los Angeles Times test kitchen: The paper has a long tradition of recipe testing, dating to the early 20th century. (Kirk McKoy/LOS ANGELES TIMES)
January 17, 2012

And that’s why we test

The story out of Chile was enough to make a publisher’s blood turn colder than a liquid nitrogen milkshake. In late December, the country’s Supreme Court ruled that the newspaper La Tercera had to shell out more than $160,000 to 13 victims who were burned while trying to fry churros in hot oil.

The readers had been following a recipe, published in the paper in 2004, that apparently called for the oil to be heated to 482 degrees, well above the smoke point for most fats. When handing down its decision, the Supreme Court said, according to published accounts, that La Tercera had failed to fully test the recipe, causing the rolls of dough to become, essentially, projectile objects.

The story tends to makes people snicker, but the account underlines a potential vulnerability in the world of newspapers as the industry continues to shrink: Can food editors — and the managers who control their budgets — still afford to test recipes with the same rigor as in the past?


The test kitchen at Sunset magazine in Menlo Park, Calif., as it appeared in 1985 — a far cry from most newspaper test kitchens at the time. It was remodeled in 1998. (Sunset magazine)

The question is important not only to avoid judicial judgments such as the one in Chile, but also to maintain journalistic standards that readers rely uponas the Internet becomes more bloated with recipes, their provenance often unknown and their accuracy questionable.

The money required to test recipes is probably not going to drain a paper’s budget. The food editors who were willing to share their figures — even ballpark ones — threw out numbers ranging from about $1,500 a month (San Francisco Chronicle) to $200 to $700 a week in groceries (Associated Press). The Washington Post spends more than $15,000 annually on testing for the entire paper; Food section policy is to test every recipe (save for those in syndicated columns in other sections) at least once, which is done in home kitchens by Food staffers, certain Food columnists and volunteer testers.

Perhaps surprising, many of the editors contacted for this story said their recipe-testing budgets have not been targeted for reduction or elimination — at least to their knowledge. What might be more surprising, however, is that some of them had to persuade their bosses to institute recipe testing in the first place. It would seem that the era of recipe testing at newspapers is a more recent invention; old timers recall when food editors in the 1980s published recipes straight from publicists and major food manufacturers, no questions asked.

Primary sources

San Francisco Chronicle Food and Wine editor Michael Bauer remembers trying to persuade his supervisors at the Kansas City Star in 1980, when he became food editor there, to start testing recipes. He recalls attempting to appeal to their sense of journalistic integrity.

“As a journalist, you always go to the source, and testing recipes is a source,” Bauer remembers arguing to his superiors. “Not testing them is taking second- and third-hand opinions.” He didn’t win the argument and had to foot the bill for recipe testing himself.

In 1986, Bauer landed at the San Francisco Chronicle, where he and his colleagues have built a test kitchen and culinary complex that is an envy of the newspaper industry. “It was just something I was able to get through the years,” says Bauer. “It was always done through the back door, one small step at a time.”

The Chronicle’s food and wine section is housed in a separate space behind the paper’s main building. Bauer has a wine cellar, a cookbook library and even a rooftop garden (complete with apiary). Next to Bauer’s office is the spacious test kitchen, where every one of the seven to 10 recipes published weekly is tested at least once,  if only to prepare the dish for a photo shoot.

“I think, over the years, we’ve become more stringent in recipe testing,” Bauer says. “It’s kind of a way to distinguished yourself, with the Internet and everything.”

In contrast to Bauer, Russ Parsons, veteran food editor at the Los Angeles Times, joined a paper with a long tradition of recipe testing, dating to the early 20th century. These days, the Times has a sprawling test kitchen overseen by manager Noelle Carter, a professionally trained chef. The kitchen is used not only for recipe testing (most recipes are tested about three times) but also for online videos and cooking segments for a sister television station.

“In the grand scale of things, it’s expensive, but you only lose a reader once,” says Parsons, who declined to provide his budget numbers. “My argument is, we still fact-check addresses and phone numbers, and that’s expensive, too,” in terms of staff hours.

If the two California papers are the gold standard for recipe testing, despite deep layoffs at both operations, there are plenty of other publications that reflect the harsher reality of journalism. Jill Wendholt Silva is the food editor and restaurant critic for the Kansas City Star. Things have gotten better since Bauer left: The paper now has a humble test kitchen. But the Star has only one food staffer: Jill Wendholt Silva.

“You take on as much as you can,” Wendholt Silva says. “You weigh how much this recipe needs to be tested versus other things you have to do.”

Test questions

Wendholt Silva, like some others in her position, has had to develop a sort of loose calculation for when to test and when not to test. The Star editor, for example, rarely bothers to test the Come Into My Kitchen column, which features reader-submitted recipes. “We don’t test every single one of those,” she says, because “they’ve made it over and over and over again.”

Both Wendholt Silva and Lee Svitak Dean, food editor of the Star Tribune in Minneapolis, have learned to scour the wires for recipes that are worthy of publishing without testing, such as those printed in magazines or newspapers that already test dishes.

“It would be rare to test the wire recipes that we use as filler, as we don’t have the staff to do that,” says Dean, one of two employees in the Taste section, which publishes three to six recipes a week. “With wire pieces, I’m very careful about who is the author. I make it a point of choosing people . . . or using individuals that I trust.”

Despite layoffs and budget cuts, the drive to publish tested recipes remains strong in the newspaper industry. “It’s one of the most vital things that we can provide,” says Joe Gray, food editor of the Chicago Tribune, who says the paper tests recipes in staff-generated stories but not always those in wire-service articles. “We can break things down and explain them. I find that a lot of Web sites don’t deliver that.”

Even with all of the precautions, however, mistakes get through. Parsons at the Los Angeles Times recalls a 1998 recipe for Vanilla-Baked Apples With Bourbon Sauce that had been tested several times. Shortly after publication, a reader contacted Parsons to say that when the foil was removed from the bourbon-baked apples in the oven, there was a small explosion. Investigating, Parsons discovered that modern ovens with tight seals will hold in alcohol vapors, potentially causing a heat flash when the vapors are exposed to oxygen.

The Times corrected the recipe — and survived the scare. “Thank God we’re not in Chile,” Parsons deadpanned.

Wire service

To help newspapers avoid disasters, J.M. Hirsch pushed the Associated Press to adopt recipe testing when he became food editor at the wire service about seven years ago. “At the time, we weren’t really testing anything,” he says.

Hirsch has changed that. He has hired two trained chefs to work with him in his home kitchen, which has been expanded and modernized, to test and photograph every single recipe that goes out over the wire. Hirsch considers it a service both to readers and to the 1,700-plus newspapers that subscribe to the AP in the United States. “If the recipe is wrong, they hear about it, because their local readers call them and complain,” Hirsch says.

Though he can’t say exactly how many more subscribers are turning more to the AP for recipes as the industry suffers, Hirsch suspects the numbers are increasing. “Their budgets are shrinking, and they’re relying on us more to fill those gaps,” says Hirsch, who tries to publish five to seven recipes weekly.

If the newspaper industry understands the importance of recipe testing in the Internet age, it is not alone. Some who call the Web home are introducing the same testing rigors to the online world. “I don’t trust much of what I find on the Internet,” says Kenji Lopez-Alt, a former editor at Cook’s Illustrated who is now managing editor for SeriousEats.com.

At Serious Eats, Lopez-Alt has tried to encourage the same exacting spirit, if not the same protocols, found at Cook’s Illustrated, where every recipe is tested on average about 30 times, he says. Lopez-Alt might test variations of a recipe for his Food Lab column 30 times, but elsewhere on the site, he’s content to see one test of a published recipe.

“Our recipe program is more focused on good testing and limiting the recipes that we run to those that we have tested ourselves,” he says. The way Lopez-Alt talks, he sounds like an online equivalent of a Michael Bauer or other like-minded editors who are committed to makingrecipes approachable for home cooks. It would seem the newspaper industry might have yet another thing to worry about.

by Tim Carman

The story out of Chile was enough to make a publisher’s blood turn colder than a liquid nitrogen milkshake. In late December, the country’s Supreme Court ruled that the newspaper La Tercera had to shell out more than $160,000 to 13 victims who were burned while trying to fry churros in hot oil.

The readers had been following a recipe, published in the paper in 2004, that apparently called for the oil to be heated to 482 degrees, well above the smoke point for most fats. When handing down its decision, the Supreme Court said, according to published accounts, that La Tercera had failed to fully test the recipe, causing the rolls of dough to become, essentially, projectile objects.

The story tends to makes people snicker, but the account underlines a potential vulnerability in the world of newspapers as the industry continues to shrink: Can food editors — and the managers who control their budgets — still afford to test recipes with the same rigor as in the past?

The question is important not only to avoid judicial judgments such as the one in Chile, but also to maintain journalistic standards that readers rely uponas the Internet becomes more bloated with recipes, their provenance often unknown and their accuracy questionable.

The money required to test recipes is probably not going to drain a paper’s budget. The food editors who were willing to share their figures — even ballpark ones — threw out numbers ranging from about $1,500 a month (San Francisco Chronicle) to $200 to $700 a week in groceries (Associated Press). The Washington Post spends more than $15,000 annually on testing for the entire paper; Food section policy is to test every recipe (save for those in syndicated columns in other sections) at least once, which is done in home kitchens by Food staffers, certain Food columnists and volunteer testers.

Perhaps surprising, many of the editors contacted for this story said their recipe-testing budgets have not been targeted for reduction or elimination — at least to their knowledge. What might be more surprising, however, is that some of them had to persuade their bosses to institute recipe testing in the first place. It would seem that the era of recipe testing at newspapers is a more recent invention; old timers recall when food editors in the 1980s published recipes straight from publicists and major food manufacturers, no questions asked.

Primary sources

San Francisco Chronicle Food and Wine editor Michael Bauer remembers trying to persuade his supervisors at the Kansas City Star in 1980, when he became food editor there, to start testing recipes. He recalls attempting to appeal to their sense of journalistic integrity.

“As a journalist, you always go to the source, and testing recipes is a source,” Bauer remembers arguing to his superiors. “Not testing them is taking second- and third-hand opinions.” He didn’t win the argument and had to foot the bill for recipe testing himself.

In 1986, Bauer landed at the San Francisco Chronicle, where he and his colleagues have built a test kitchen and culinary complex that is an envy of the newspaper industry. “It was just something I was able to get through the years,” says Bauer. “It was always done through the back door, one small step at a time.”

The Chronicle’s food and wine section is housed in a separate space behind the paper’s main building. Bauer has a wine cellar, a cookbook library and even a rooftop garden (complete with apiary). Next to Bauer’s office is the spacious test kitchen, where every one of the 15 to 20 recipes published weekly is tested at least once,  if only to prepare the dish for a photo shoot.

“I think, over the years, we’ve become more stringent in recipe testing,” Bauer says. “It’s kind of a way to distinguished yourself, with the Internet and everything.”

In contrast to Bauer, Russ Parsons, veteran food editor at the Los Angeles Times, joined a paper with a long tradition of recipe testing, dating to the early 20th century. These days, the Times has a sprawling test kitchen overseen by manager Noelle Carter, a professionally trained chef. The kitchen is used not only for recipe testing (most recipes are tested about three times) but also for online videos and cooking segments for a sister television station.

“In the grand scale of things, it’s expensive, but you only lose a reader once,” says Parsons, who declined to provide his budget numbers. “My argument is, we still fact-check addresses and phone numbers, and that’s expensive, too,” in terms of staff hours.

If the two California papers are the gold standard for recipe testing, despite deep layoffs at both operations, there are plenty of other publications that reflect the harsher reality of journalism. Jill Wendholt Silva is the food editor and restaurant critic for the Kansas City Star. Things have gotten better since Bauer left: The paper now has a humble test kitchen. But the Star has only one food staffer: Jill Wendholt Silva.

“You take on as much as you can,” Wendholt Silva says. “You weigh how much this recipe needs to be tested versus other things you have to do.”

Test questions

Wendholt Silva, like some others in her position, has had to develop a sort of loose calculation for when to test and when not to test. The Star editor, for example, rarely bothers to test the Come Into My Kitchen column, which features reader-submitted recipes. “We don’t test every single one of those,” she says, because “they’ve made it over and over and over again.”

Both Wendholt Silva and Lee Svitak Dean, food editor of the Star Tribune in Minneapolis, have learned to scour the wires for recipes that are worthy of publishing without testing, such as those printed in magazines or newspapers that already test dishes.

“It would be rare to test the wire recipes that we use as filler, as we don’t have the staff to do that,” says Dean, one of two employees in the Taste section, which publishes three to six recipes a week. “With wire pieces, I’m very careful about who is the author. I make it a point of choosing people . . . or using individuals that I trust.”

Despite layoffs and budget cuts, the drive to publish tested recipes remains strong in the newspaper industry. “It’s one of the most vital things that we can provide,” says Joe Gray, food editor of the Chicago Tribune, who says the paper tests recipes in staff-generated stories but not always those in wire-service articles. “We can break things down and explain them. I find that a lot of Web sites don’t deliver that.”

Even with all of the precautions, however, mistakes get through. Parsons at the Los Angeles Times recalls a 1998 recipe for Vanilla-Baked Apples With Bourbon Sauce that had been tested several times. Shortly after publication, a reader contacted Parsons to say that when the foil was removed from the bourbon-baked apples in the oven, there was a small explosion. Investigating, Parsons discovered that modern ovens with tight seals will hold in alcohol vapors, potentially causing a heat flash when the vapors are exposed to oxygen.

The Times corrected the recipe — and survived the scare. “Thank God we’re not in Chile,” Parsons deadpanned.

Wire service

To help newspapers avoid disasters, J.M. Hirsch pushed the Associated Press to adopt recipe testing when he became food editor at the wire service about seven years ago. “At the time, we weren’t really testing anything,” he says.

Hirsch has changed that. He has hired two trained chefs to work with him in his home kitchen, which has been expanded and modernized, to test and photograph every single recipe that goes out over the wire. Hirsch considers it a service both to readers and to the 1,700-plus newspapers that subscribe to the AP in the United States. “If the recipe is wrong, they hear about it, because their local readers call them and complain,” Hirsch says.

Though he can’t say exactly how many more subscribers are turning more to the AP for recipes as the industry suffers, Hirsch suspects the numbers are increasing. “Their budgets are shrinking, and they’re relying on us more to fill those gaps,” says Hirsch, who tries to publish five to seven recipes weekly.

If the newspaper industry understands the importance of recipe testing in the Internet age, it is not alone. Some who call the Web home are introducing the same testing rigors to the online world. “I don’t trust much of what I find on the Internet,” says Kenji Lopez-Alt, a former editor at Cook’s Illustrated who is now managing editor for SeriousEats.com.

At Serious Eats, Lopez-Alt has tried to encourage the same exacting spirit, if not the same protocols, found at Cook’s Illustrated, where every recipe is tested on average about 30 times, he says. Lopez-Alt might test variations of a recipe for his Food Lab column 30 times, but elsewhere on the site, he’s content to see one test of a published recipe.

“Our recipe program is more focused on good testing and limiting the recipes that we run to those that we have tested ourselves,” he says. The way Lopez-Alt talks, he sounds like an online equivalent of a Michael Bauer or other like-minded editors who are committed to makingrecipes approachable for home cooks. It would seem the newspaper industry might have yet another thing to worry about.

Tim Carman serves as the full-time writer for the Post's Food section and as the $20 Diner for the Weekend section, a double duty that requires ingesting more calories than a draft horse.
SECTION: {section=lifestyle, subsection=food}!!!
INITIAL commentConfig: {includereply=true, canvas_permalink_id=washpost.com/8bvh5zpd9k, allow_comments=true, commentmaxlength=2000, includeshare=true, display_comments=true, canvas_permalink_app_instance=m6yzjj840m, display_more=true, moderationrequired=false, includefeaturenotification=true, canvas_allcomments_id=washpost.com/km4ey0dajm, comments_period=14, defaultsort=reverseChronological, includevoteofftopic=false, allow_videos=false, includesorts=true, markerdisplay=post_commenter:Post Commenter|staff:Post Writer|top_commenter:Post Forum|top_local:Washingtologist|top_sports:SuperFan|fact_checker:Fact Checker|post_recommended:Post Recommended|world_watcher:World Watcher|cultuer_connoisseur:Culture Connoisseur|weather_watcher:Capital Weather Watcher|post_contributor:Post Contributor, childrenitemsperpage=3, includeheader=true, includeverifiedcommenters=true, defaulttab=all, includerecommend=true, includereport=true, maxitemstop=3, source=washpost.com, allow_photos=false, maxitems=15, display_ugc_photos=false, includepause=true, canvas_allcomments_app_instance=6634zxcgfd, includepermalink=false}!!!

UGC FROM ARTICLE: {allow_comments=true, allow_photos=false, allow_videos=false, comments_period=14, comments_source=washpost.com, default_sort=, default_tab=, display_comments=true, is_ugc_gallery=false, max_items_to_display=15, max_items_to_display_top=3, moderation_required=false, stream_id=}!!!

FINAL commentConfig: {includereply=true, canvas_permalink_id=washpost.com/8bvh5zpd9k, allow_comments=true, commentmaxlength=2000, includeshare=true, display_comments=true, canvas_permalink_app_instance=m6yzjj840m, display_more=true, moderationrequired=false, includefeaturenotification=true, canvas_allcomments_id=washpost.com/km4ey0dajm, comments_period=14, defaultsort=reverseChronological, includevoteofftopic=false, allow_videos=false, includesorts=true, markerdisplay=post_commenter:Post Commenter|staff:Post Writer|top_commenter:Post Forum|top_local:Washingtologist|top_sports:SuperFan|fact_checker:Fact Checker|post_recommended:Post Recommended|world_watcher:World Watcher|cultuer_connoisseur:Culture Connoisseur|weather_watcher:Capital Weather Watcher|post_contributor:Post Contributor, childrenitemsperpage=3, includeheader=true, includeverifiedcommenters=true, defaulttab=all, includerecommend=true, includereport=true, maxitemstop=3, source=washpost.com, allow_photos=false, maxitems=15, display_ugc_photos=false, includepause=true, canvas_allcomments_app_instance=6634zxcgfd, includepermalink=false}!!
Comments
SECTION: {section=lifestyle, subsection=food}!!!
INITIAL commentConfig: {includereply=true, canvas_permalink_id=washpost.com/8bvh5zpd9k, allow_comments=true, commentmaxlength=2000, includeshare=true, display_comments=true, canvas_permalink_app_instance=m6yzjj840m, display_more=true, moderationrequired=false, includefeaturenotification=true, canvas_allcomments_id=washpost.com/km4ey0dajm, comments_period=14, defaultsort=reverseChronological, includevoteofftopic=false, allow_videos=false, includesorts=true, markerdisplay=post_commenter:Post Commenter|staff:Post Writer|top_commenter:Post Forum|top_local:Washingtologist|top_sports:SuperFan|fact_checker:Fact Checker|post_recommended:Post Recommended|world_watcher:World Watcher|cultuer_connoisseur:Culture Connoisseur|weather_watcher:Capital Weather Watcher|post_contributor:Post Contributor, childrenitemsperpage=3, includeheader=true, includeverifiedcommenters=true, defaulttab=all, includerecommend=true, includereport=true, maxitemstop=3, source=washpost.com, allow_photos=false, maxitems=15, display_ugc_photos=false, includepause=true, canvas_allcomments_app_instance=6634zxcgfd, includepermalink=false}!!!

UGC FROM ARTICLE: {allow_comments=true, allow_photos=false, allow_videos=false, comments_period=14, comments_source=washpost.com, default_sort=, default_tab=, display_comments=true, is_ugc_gallery=false, max_items_to_display=15, max_items_to_display_top=3, moderation_required=false, stream_id=}!!!

FINAL commentConfig: {includereply=true, canvas_permalink_id=washpost.com/8bvh5zpd9k, allow_comments=true, commentmaxlength=2000, includeshare=true, display_comments=true, canvas_permalink_app_instance=m6yzjj840m, display_more=true, moderationrequired=false, includefeaturenotification=true, canvas_allcomments_id=washpost.com/km4ey0dajm, comments_period=14, defaultsort=reverseChronological, includevoteofftopic=false, allow_videos=false, includesorts=true, markerdisplay=post_commenter:Post Commenter|staff:Post Writer|top_commenter:Post Forum|top_local:Washingtologist|top_sports:SuperFan|fact_checker:Fact Checker|post_recommended:Post Recommended|world_watcher:World Watcher|cultuer_connoisseur:Culture Connoisseur|weather_watcher:Capital Weather Watcher|post_contributor:Post Contributor, childrenitemsperpage=3, includeheader=true, includeverifiedcommenters=true, defaulttab=all, includerecommend=true, includereport=true, maxitemstop=3, source=washpost.com, allow_photos=false, maxitems=15, display_ugc_photos=false, includepause=true, canvas_allcomments_app_instance=6634zxcgfd, includepermalink=false}!!
Show Comments