One of the most talked-about public-health initiatives is improving indoor air quality in the rural developing world. Traditional cookstoves — mud basins in which villagers burn wood, charcoal or dung — are the main obstacle. The fire releases particulate matter that contributes to pneumonia, lung cancer and heart disease, among many other maladies.

The problems disproportionately affect women, who do most of the cooking in this population, and the children who are often nearby. The World Health Organization estimates that indoor air pollution kills
2 million people every year, about as many as malaria and tuberculosis combined, although this number is in some dispute.

Over the past two years, the United States has pledged
$105 million
to fighting the cookstove problem. Secretary of State Hillary Clinton has repeatedly spoken about it. And of course, no global health issue is complete without a Hollywood advocate; in this case, it’s Julia Roberts.

The main goal of the effort is to replace traditional cookstoves with “clean cookstoves,” which are more efficient and often feature chimneys to direct fumes out of the home.

An ambitious goal

The Global Alliance for Clean Cookstoves, which the U.S. government helped found in 2010, aims to help 100 million households replace their stoves with clean alternatives by 2020. But there’s a potential problem with this ambitious goal: A new study suggests the replacement stoves may not make any difference.

An Iraqi woman bakes bread using a mud stove in Baghdad. (Akram Saleh/GETTY IMAGES)

Clean cookstove advocates have based their strategy on two kinds of studies. In observational studies, researchers compare the prevalence of pneumonia, lung cancer, low birthweight, etc., in traditional-cookstove houses with rates in houses that use clean cookstoves. That’s a good place to start, but it’s subject to all sorts of confounders. Families with better cookstoves are likely to have more money, better nutrition and superior access to health care. So the differences in health status may have little or nothing to do with the stoves.

There has also been a single large-scale prospective study, known as RESPIRE. (The name stands for Randomized Exposure Study of Pollution Indoors and Respiratory Effects.) The study showed that clean cookstoves improved air quality and health, although not nearly as much as was suggested in the observational studies.

There are several reasons, though, to question RESPIRE’s results. First, the brick and metal stoves tested in its study were the Cadillacs of clean cookstoves. They cost $100 to $150 apiece, which is more than the average annual per capita income in many poor villages. Rolling out these stoves to
100 million homes would be hugely expensive.

More problematically, the conditions of the RESPIRE study probably weren’t realistic. Field workers trained families on the use and care of the test stoves, and returned weekly to make sure the stoves were being used properly. When the stoves broke down, the field workers arranged for repair. Such intensive intervention would be nearly impossible on a large scale.

Studies challenging the ability of clean stoves to improve health and environment are beginning to come in. In March, researchers from India and California published a study showing that certain clean cookstove models occasionally release a larger volume of certain pollutants than the traditional stoves they’re intended to replace.

‘Up in Smoke’

Three American researchers released a randomized, controlled study Monday that is even more damaging to the clean cookstove movement. In their study, entitled “Up in Smoke,” Rema Hanna of Harvard University and Esther Duflo and Michael Greenstone of MIT sold clean cookstoves to more than 2,600 households in 44 Indian villages, then followed the families for three years.

As in RESPIRE, the stoves in this study featured a two-burner system and a chimney to direct fumes out of the house. Made primarily of mud, the stoves cost just $12.50 each; the families paid 75 cents for the stove, installation and maintenance. Training sessions were offered, but field workers didn’t continually retrain the families or call in maintenance for them.

“Up in Smoke” studied more than 10 times as many people as RESPIRE, and followed them for more than twice as long. The larger scale enabled researchers to get a sense of how stove usage changes over time, and how those changes might affect health.

The results were beyond disappointing.

By the end of the study, most families were barely using the clean cookstove at all. Owners eventually allowed their stoves to fall into disrepair, ignoring cracks that undermined the functioning of the chimney. When they did use the stoves, many used them improperly.

For example, when they were cooking in one pot, they often failed to cover the second burner. Others failed to clean the chimneys. These mistakes undercut efficiency and allowed fumes to flow into the house.

As a result, the stoves were no more efficient in practice than the traditional models they replaced. The families burned the same amount of wood, so indoor air pollution continued unabated. (From a wider environmental perspective, that means the clean stoves didn’t slow deforestation and greenhouse gas emissions, either).

Air quality measurements barely budged in the test homes. In the first year, exhaled carbon monoxide levels dropped a modest 7.5 percent in families with clean cookstoves. (Exhaled carbon monoxide is used as a proxy for particulate matter, because researchers can sample a person’s breath rather than test the air in a house.) By year two, air quality had returned to pre-study levels.

It’s not clear how much of a decrease in particulate matter is required to improve health, but the small improvement seen in the study clearly wasn’t enough. Lung functioning, incidence of respiratory illnesses, the body mass index of children in the household, and infant health outcomes such as birthweight and infant mortality, did not change significantly.

Does this mean we should give up the fight against indoor air pollution? Certainly not. Anyone can see that traditional cookstoves are unhealthful. But the study is a reminder that public health is a tricky business: Just because a solution works in a laboratory — or among a small group of closely watched test subjects — doesn’t mean it should be rolled out to 100 million households. Human behavior has to be tested as rigorously as the efficiency of hardware or indoor air quality, because it can undermine all of our efforts.

The Global Alliance for Clean Cookstoves and its partners are working to fill the gaps that remain in medical, technical and behavioral research, with further studies in Ghana, Nepal and Kenya. The International Standards Organization recently promulgated guidelines to evaluate clean cookstoves. These efforts, plus the the “Up in Smoke” study, suggest we’re not yet ready to distribute clean stoves worldwide.

As Greenstone notes, “This isn’t an argument against spending money; it’s an argument against spending money unwisely.