The Washington PostDemocracy Dies in Darkness

What the Reinhart-Rogoff debacle tells us about the mysteries of macroeconomics

Every time I get on an airplane, I find myself a little amazed that it can fly. Let me get this straight: This big metal thing with a couple hundred people inside is, by sending thrust backward, going to propel itself forward and, thanks to the shape of the wings, lift itself up to 30,000 feet in the air, delivering us safely to a distant city in a surprisingly short time? And there may be snacks along the way?

None of it would seem remotely plausible if explained to my great-great-grandfather. And yet, we know and accept that it works, and works so well that air travel is extraordinarily safe and vital to modern commerce.

Macroeconomics is surprisingly similar to jet travel in that regard. There are a lot of things that happen in the real world that seem to defy common sense. You wouldn’t think that a central bank adjusting the supply of money, by, say, cutting interest rates by 1 percent, would enable millions more people to find jobs. But so it is.

Depressions shouldn’t happen; after all, the United States was just as well-endowed with fertile land, productive factories and hard-working laborers in 1932 as it was in 1929. So there is no inherent reason the nation should have been producing far less stuff. But that was the case.

In other words, considering both modern jet travel and modern macroeconomics, it wouldn’t have done much good to ask someone living a century ago to sit down and sketch out, using theory and mathematics, how the world would end up looking. Mankind learns how these immensely complicated things work in fits and starts. You might start with theory, but you have to constantly test that theory against reality.

Which brings us to Carmen Reinhart and Kenneth Rogoff, the economists whose work on the relationship between high public debt levels and economic growth came under question this week. The pair's Excel coding error in the 2010 study has earned plenty of snark in the econ-punditsphere in the last couple of days. But the more significant critique of the Reinhart-Rogoff paper, which found that countries with debt levels above 90 percent of GDP have significantly lower growth than those below that threshold, is that they used a seemingly arbitrary system of weighting different countries and years to arrive at that result — and that other methods that seem as valid as theirs do not produce the same result.

The methodological questions boost earlier critiques of the Reinhart-Rogoff work -- that their 90 percent threshold was fairly arbitrary and that the work was frequently interpreted by pundits and politicians to explain more than it actually did. (Does high debt cause slow growth, or does slow growth cause high debt? The original paper didn’t argue either way, though deficit hawks seized on the first cause-and-effect.)

Still, I have found the glee shown by economics commentators who've pounced on Reinhart-Rogoff to be a little discomfiting. Maybe that’s because the wider scope of pair's work is so important to understanding the black box that is macroeconomics. They sifted through centuries' worth of historical materials to create a rich database on the kinds of crises that different countries have faced and when. Did you know that Belgium was hit by an inflation crisis in 1833, a banking crisis in 1838, a stock market crash in 1900, a currency crisis in 1914 and all four in 1926? I do, because I just conveniently pulled that information from the Reinhart-Rogoff Web site.

Macroeconomists have become more sophisticated at using math to develop models of how economies could, should or might work. We can test those models against the kind of information that Reinhart-Rogoff produce and catalog and determine whether they're useful.

The great challenge of macroeconomics is that it does not lend itself to experiments. The builders of early jet aircraft could develop prototypes and send test pilots up to test their results. But policymakers can’t very well take the same chances with their nations’ economies, because the consequences are much greater if they get it wrong. You can risk crashing a prototype jet airplane, but you can’t risk crashing an entire national economy.

As such, macroeconomists have to rely on natural experiments to understand how the economy really works, examining how different countries pursuing different policies at different times and in different circumstances get different results.

But there's a problem with relying on those comparisons: There are only so many countries and so many years to study (particularly if you’re limited to times of crisis). If you are making policy for the United States in 2013, it isn’t necessarily prudent to base decisions on what happened in Belgium in 1926. The sample size is small to begin with, and it can shrink to nothing when you adjust for the fact that the financial system has evolved and that large economies with their own currency face different circumstances than small economies more easily whipsawed by external fluctuations. (That’s one reason Japan is such an interesting and important country to study in understanding the current U.S. economic weakness. Its economic situation is the closest to ours.)

And that’s the original problem with the Reinhart-Rogoff study that everyone is now piling on. It was widely used to draw big conclusions about whether or when countries should pivot to austerity, even though it was a relatively thin data set..

But in the world of macro, sometimes that is the best we have. And the fact that this study was flawed shouldn’t dissuade academics from doing the hard, messy work of trying to parse the past to try to create a world in which economies crash as rarely as jumbo jets do.