The hard-working staff here at Spoiler Alerts has been interested in the independent effect of prominent ranking exercises for some time now. Scholarship by Judith Kelley, Beth Simmons and other researchers demonstrates that states care a surprising amount about how they are ranked and rated by, say, the Organization for Economic Cooperation and Development on education, the U.N. Development Program on human development, Transparency International on corruption, and so forth. Nor is this interest limited to national governments, as anyone familiar with how universities treat the U.S. News and World Report rankings on higher education can tell you. In a way, it is remarkable that institutions possessing modest capabilities can affect powerful states with something as simple as a ranking schema.
Other scholars, particularly Alex Cooley and Jack Snyder, have pointed out the dark side of such ranking exercises. Governments will try to game the system so as to make it appear that there has been improvement in, say, combating corruption, when all that is really happening is a box-checking exercise. Even more disturbing is if those doing the rankings find themselves altering the results in response to external political pressure.
Speaking of which, last week the World Bank released a bombshell external report by WilmerHale, commissioned by the bank, over alleged irregularities in its “Doing Business” (DB) index, particularly in 2018 and 2020. The report confirmed those irregularities in pretty spectacular fashion.
In particular, the report specifies how World Bank CEO Kristalina Georgieva — now the president of the International Monetary Fund — allegedly played a prominent role in fiddling with the index. The report makes clear that the initial ranking would have shown China dropping from 78th to 85th in the DB index at a sensitive time for the bank, which was trying to earn Chinese backing for a capital campaign. In essence, the report says, Georgieva ordered the “Doing Business” staff to jerry-rig its 2018 ranking so as to inflate China’s standing.
The report states that “Georgieva also chastised the Bank’s then-Country Director for mismanaging the Bank’s relationship with China and failing to appreciate the importance of the Doing Business report to the country.” The World Bank senior director in charge of the index acknowledged to WilmerHale that his leadership team had “made ‘judgment calls’ to push the data in a certain direction to accommodate geopolitical considerations” and that Georgieva “thanked him for doing his ‘bit for multilateralism.’ ”
Georgieva has denied the report’s allegations. The New York Times’ Alan Rappeport reports that she told IMF staff, “I would ask staff to please check, double-check, triple-check, but never change, never manipulate what the data tells us.” The WilmerHale report, however, flatly contradicts this assertion. When the Wall Street Journal reported that “the episode is a reputational hit for Ms. Georgieva,” it was going for understatement.
On the heels of the report, the World Bank announced that it was discontinuing its “Doing Business” index. This rendered a proposed reform of the DB index by an external review group that actually made a lot of sensible suggestions, including an overhaul of the index’s methodology. The Center for Global Development’s Justin Sandefur, who helped draft the overhaul proposal, thinks the discontinuation makes sense: “DB was simply too ad hoc, too subjective, and too prone to political interference, that it had become a significant drag on the credibility of World Bank research.”
Sandefur has a valid point, and eliminating the temptation by senior World Bank officials to compromise their principles is probably a good thing. The cynic in me, however, does worry about the long-term implications of this move. As more competitive authoritarian states show willingness to pressure those who rank, I worry whether there will be a Gresham’s law of ranking schema. If organizations that care about their standing get out of this game, others with less scruples will step in. There will always be a demand for these rankings from folks who like information shortcuts, They are not going to go away simply because some organizations choose to sit on the sidelines.