What is a scorecard?
A scorecard is a way of rating or ranking how countries or other actors perform in a given policy area. These scorecards are not one-off rankings; they recur, usually yearly.
Why should states, or anyone else, care about scorecards? First of all, they are easier to understand and digest than complicated policy reports. Instead of emphasizing detailed data, they sort countries into categories (e.g., countries that are succeeding vs. countries that are failing), or rank them with some score, showing which countries are at the top and at the bottom. These categories and rankings are framed to pressure the countries being ranked. For example, if your country is at the bottom of a well-respected scorecard for “Ease of Doing Business,” you might find that international businesses start to avoid investing in your economy.
Beth Simmons and I have data on more than 180 scorecards set up by countries, international organizations and nongovernmental organizations. More and more scorecards have been appearing, covering more and more topics. The U.S. government has scorecards for areas as varied as aid, religious freedom, narcotics control, child labor and international property rights protection.
This proliferation of scorecards has brought more and more critiques. Sometimes the scorecards are based on questionable data, or questionable ways of organizing the data. Yet, even if they are biased or based on dubious assumptions, scorecards do influence behavior. Why?
Here’s why these scorecards are more than just scraps of paper
My recent book on the TIP report on human trafficking explains what I call the “cycle of scorecard diplomacy.” The TIP report doesn’t just rank countries. Producing the report involves U.S. diplomats on the ground engaging with governments year-round and orchestrating indirect pressure by media and civil society.
These combined efforts make governments concerned about their ratings. Officials react strongly to the report, particularly to the tier that their country falls into. U.S. diplomats wrote in a cable to Washington that one Albanian official’s “face went pale when told” of his country’s low rating. That’s hard to fake! Many governments criticize the report in public but cooperate with the U.S. in private. This suggests that these grades are really getting to governments. Even allies such as Israel feel the sting. In 2009, Deputy Foreign Minister Danny Ayalon told the Knesset subcommittee analyzing the TIP report that:
A U.S. government report lumping Israel together with states such as Afghanistan, Jordan and Botswana in its success in combating human trafficking has troubling political implications for Israel. … It has a direct impact on Israel’s standing in the international community.
Because countries are rated again and again, they have an incentive to improve their behavior in the hopes of boosting future grades. As a result, states pay more sustained attention to an issue than they would do if they were just shamed in an ad hoc way.
Simmons and I have shown that countries criminalize human trafficking more quickly when they are included in the report, get worse grades or see their grades drop. My work on TIP shows that this is not just because countries fear being sanctioned. The stigma of the scorecard makes states change their behavior. Countries that criminalize trafficking also work harder on related efforts to fight the problem. In many countries, the TIP report has led states to set up new institutions, to train judges and police, to improve shelters, and to increase trafficking prosecutions and convictions. Thus scorecards can prompt real changes.
But scorecards have their limits
None of this is to say that scorecards are universally successful. Critics of scorecards are correct when they say, for instance, that scorecards are often ideologically loaded and based on dubious statistics. Sometimes countries try to game the rankings rather than make genuine improvements, as Belarus has tried with TIP. When these problems combine, they may even be dangerous because they encourage unwise or wasteful reforms.
Still, all effective tools can be dangerous if used badly. The TIP report has been criticized for inconsistency, shifting goal posts and U.S. arrogance. Yet in a global survey that I did with Andrew Heiss of nearly 500 anti-trafficking organizations, more than 60 said they think the United States is an important actor in the countries where they work, with more than a quarter saying that the United States is the most important actor. Amazingly, only two organizations had a negative view. And people working for international organizations have told me that they appreciate the report enormously.
It’s hard for international organizations to criticize states frankly over sensitive problems, since they rely on those governments for funding, support and cooperation. At their best, scorecards can hold governments accountable and help spread solutions. That’s why it’s important to understand exactly how they can do this and when they succeed or fail.
Judith Kelley is the Kevin D. Gorter professor of public policy and political science and a senior associate dean at the Sanford School of Public Policy at Duke University. Her book Scorecard Diplomacy: Grading States to Influence their Reputation and Behavior is recently out from Cambridge University Press.