One of the more influential studies that's often used to argue for austerity has come in for an extensive new critique. (Update: The authors respond down below.)
Now, those are two methodological objections. But there's also a third problem, as Mike Konczal details here. Reinhart and Rogoff appear to have made an error with one of their Excel spreadsheet formulas. By typing AVERAGE(L30:L44) at one point instead of AVERAGE(L30:L49), they left out Belgium, a key counterexample:
This discrepancy wasn't caught earlier because Reinhart and Rogoff hadn't made their full underlying data public. They only shared their spreadsheet with the Herndon, Ash and Pollin after the latter three tried to replicate the initial results and failed.
Taken together, those three changes lead to a different analysis. Herndon, Ash, and Pollin conclude that "the average real GDP growth rate for countries carrying a public debt-to-GDP ratio of over 90 percent is actually 2.2 percent, not -0.1 percent as as published in Reinhart and Rogoff."
It's worth emphasizing, however, that the Excel coding error itself is only responsible for a small portion of this difference in results — about 0.3 percentage points. So the spreadsheet mistake, by itself, does not appear to be fatal. Here's the difference between the two studies in chart form:
A few extra points:
--This is not the first critique of Reinhart-Rogoff. As Dylan Matthews explained here, other economists have argued that the initial paper got the causality backward. Countries have high debt-to-GDP ratios because they have slow growth, rather than the other way around. This is a far more substantive criticism than the debate over the precise numbers above.
--Reinhart has responded to the critique of causality this way: "We’re quite aware that you have causality going in both directions,” she told Dylan. “But please point out to me what episodes from 1800 to the present have we had advanced economies who carried high levels of debt growing as rapidly or more rapidly than the norm..”
--At the same time, Reinhart-Rogoff isn't the only study finding that high debt levels are associated with low growth. A 2010 paper (pdf) from the IMF, for instance, found that "a 10 percentage point increase in the initial debt-to-GDP ratio is associated with a slowdown in annual real per capita GDP growth of around 0.2 percentage points per year, with the impact being somewhat smaller in advanced economies." Still, similar questions about causality apply to these papers too.
--Tyler Cowen has some additional thoughts here. He tentatively agrees with the critiques of the paper, but suggests that Reinhart and Rogoff's influence on the austerity debate has been overstated: "The 'case for austerity' didn’t rest much on R&R in the first place, rather on the notion that the bills have to be paid, dawdling on adjustment is not always so easy, and the feasible sum of international redistribution is quite low."
Update: Reinhart and Rogoff have now responded. I've bolded the key parts:
We literally just received this draft comment, and will review it in due course. On a cursory look, it seems that that Herndon Ash and Pollin also find lower growth when debt is over 90% (they find 0-30 debt/GDP , 4.2% growth; 30-60, 3.1 %; 60-90, 3.2%,; 90-120, 2.4% and over 120, 1.6%). These results are, in fact, of a similar order of magnitude to the detailed country by country results we present in table 1 of the AER paper and to the median results in Figure 2. And they are similar to estimates in much of the large and growing literature, including our own attached August 2012 Journal of Economic Perspectives paper (joint with Vincent Reinhart). However, these strong similarities are not what these authors choose to emphasize.The 2012 JEP paper largely anticipates and addresses any concerns about aggregation (the main bone of contention here), The JEP paper not only provides individual country averages (as we already featured in Table 1 of the 2010 AER paper) but it goes further and provide episode by episode averages. Not surprisingly, the results are broadly similar to our original 2010 AER table 1 averages and to the median results that also figure prominently. It is hard to see how one can interpret these tables and individual country results as showing that public debt overhang over 90% is clearly benign.The JEP paper with Vincent Reinhart looks at all public debt overhang episodes for advanced countries in our database, dating back to 1800. The overall average result shows that public debt overhang episodes (over 90% GDP for five years or more) are associated with 1.2% lower growth as compared to growth when debt is under 90%. (We also include in our tables the small number of shorter episodes.) Note that because the historical public debt overhang episodes last an average of over 20 years, the cumulative effects of small growth differences are potentially quite large. It is utterly misleading to speak of a 1% growth differential that lasts 10-25 years as small.By the way, we are very careful in all our papers to speak of “association” and not “causality” since of course our 2009 book THIS TIME IS DIFFERENT showed that debt explodes in the immediate aftermath of financial crises. This is why we restrict attention to longer debt overhang periods in the JEP paper., though as noted there are only a very limited number of short ones. Moreover, we have generally emphasized the 1% differential median result in all our discussions and subsequent writing, precisely to be understated and cautious, and also in recognition of the results in our core Table 1 (AER paper).Lastly, our 2012 JEP paper cites papers from the BIS, IMF and OECD (among others) which virtually all find very similar conclusions to original findings, albeit with slight differences in threshold, and many nuances of alternative interpretation. These later papers, by they way, use a variety of methodologies for dealing with non-linearity and also for trying to determine causation. Of course much further research is needed as the data we developed and is being used in these studies is new. Nevertheless, the weight of the evidence to date –including this latest comment -- seems entirely consistent with our original interpretation of the data in our 2010 AER paper.