Stop guessing whether a bill will work. Instead, let’s test it.

I’m a policy reporter. My job is to explain to my readers whether smaller class sizes help students learn, whether tax cuts boost economic growth and whether housing programs help families escape poverty.

In a perfect world, what I do would be a kind of science reporting. Just as my colleagues at the health desk often explain which medicines are effective and which are a bust, I’d ideally be able to describe what sociologists, economists and political scientists have discovered about which policies work.

More from Outlook

In Cleveland, the power of neighbors

In Cleveland, the power of neighbors

Do we know those living around us?

Five myths about missing children

Five myths about missing children

Are most abducted by strangers?

A daughter’s love from a distance

A daughter’s love from a distance

Will Mom’s illness bring us closer?

Gallery

With a few exceptions, however, I can’t really do that — at least not with the precision my health colleagues often can. More important, neither can policymakers in Congress and in many regulatory agencies. The Food and Drug Administration has more information available in deciding whether to approve a treatment that a few thousand people will receive than Congress does in considering a bill that will affect every American.

Each year, hundreds of carefully controlled, double-blind studies are done to learn whether a given pill is better than a placebo or whether a new surgery leads to quicker recoveries. Many of these studies are funded by a single agency, the National Institutes of Health. By contrast, in a typical year, no such studies are conducted to evaluate social policy proposals.

That’s not because such studies are impossible. In 1962, researchers in a small Michigan school district randomly selected 58 3- and 4-year-olds to enroll in a preschool program, then spent decades comparing them with a control group of 65 kids who didn’t go to preschool. Those who enrolled learned more and made more money as adults. In 1976, the Chicago Housing Authority randomly placed public-housing residents in apartments either in the city or in the suburbs, and then tracked the two groups. Those given places in the suburbs did better on every metric from household income to their children’s rates of college attendance.

Those studies had a big impact. The Chicago study, for example, is the main research cited in proposals to provide housing vouchers to poor families to break up pockets of “concentrated disadvantage.”

But studies like those are very rare. They’re expensive, which discourages universities and school districts (such as the one in Michigan conducting the preschool study) from doing them. And often, as in the Chicago case, they come about only because a court orders them .

Because of this rarity, it’s easy to pick nits. The preschool study involved only children with low IQs, critics noted. Maybe the results would have been different with children of average or above-average intelligence. The Chicago housing study was conducted during a time when crime was much worse than today. Maybe with safer inner cities, you wouldn’t see similar gains from sending families to the suburbs.

Researchers have spent gallons of ink arguing over such caveats, trying to figure out what can and can’t be inferred from the meager pool of good data with which they’re forced to work. At no point does the straightforward solution present itself: run another study.

Loading...

Comments

Add your comment
 
Read what others are saying About Badges