HF: What is the U.S. census, and what exactly does it do?
KP: I start with the decadal census, which asks seven questions about every household in the nation. This every-10-year count is constitutionally mandated; it apportions the 435 members of the House of Representatives to the 50 states, proportionate to each state’s share of the total population. Since 1790, the first census, state populations have unevenly increased, and at times decreased. Repeated reallocation is fundamental to the fairness of our representative democracy.
The census also includes the American Community Survey (ACS), which by law is an extension of the basic decadal census. This survey is approximately 60 congressionally mandated questions that underpin government programs — ranging from where to place and how to staff veteran’s hospitals to emergency planning by every town and city in the country, from public health programs to designing transportation networks. The ACS is the country’s largest survey (after the decadal census), reaching enough people to provide great geographic detail — e.g., an accurate description of the characteristics — age structure for instance — of an area with as few as 15,000 residents. This detail is used by states and city educational systems when building new schools and by the policy and fire departments to deploy the right equipment when rushing to an emergency.
HF: Budget disagreements and arguments over the next decadal census seem to have led to the recent resignation of the census’s director. Can you explain what happened?
KP: The director, John Thompson, basically led the 2000 census effort, when I was the overall Census Bureau director. I know him to be an excellent manager, a first-rate statistician, and one of only a handful of people in the country equipped for the very difficult challenge Congress mandated for the 2010 census: Do not spend more on the 2020 Census than was spent in 2010. This is the first time in census history when such a restriction was mandated. The only way to meet this requirement was to innovate, based on new technologies and greater use of administrative data when people do not return their form. John’s design met this challenge, but then the budget tightened further — especially funds to test innovative procedures — and there were doubts about funding a professional advertising campaign. Testing and advertising are absolutely necessary to have any chance of a successful census in 2020.
Funds were not provided on the schedule and at the scale anticipated, though even as recently as two weeks ago, I had lunch with John and he was upbeat about where they were. His resignation, then, is a surprising and disappointing development for the country. I have no direct report on the reasons but I know John to be an honorable professional. If he felt he did not have the backing of the secretary, his decision would be to resign and make way for the secretary to make his own choice of a new director, early enough in the process for that person to adjust the census design in line with the level of funding and when it can be expected.
HF: What is a good census?
KP: A good census counts everyone in the country, counts each only once, and counts all in the right place. Any census has both an overcount and an undercount — the question is the magnitude. An overcount results in some geographic areas and groups receiving more benefits than they deserve, because people are being double counted. For example, people might end up being accidentally counted at both their summer home and their primary residence. An undercount, obviously, disproportionately penalizes the affected area or group.
The undercount has always been the greater problem. George Washington complained when the 1790 census counted just under 4 million. He wanted it at more than 4 million, as a signal to the British that the new country was robust and rapidly growing — the British should not think of trying to reconquer the colonies.
The undercount was anecdotally discussed with every census, but, until 1942, there was no adequate way to measure it. The obligatory draft registration for all males between the ages of 18 and 34 produced a count of that demographic group that could be compared to the census count. Moreover, because the armed forces were segregated, it was possible to calculate any differences in the undercount between black and white males. Black Americans were abut three times as likely to be undercounted as white Americans. That started a long and challenging effort to reduce the undercount, and especially the problem that it was differential by race.
Keep in mind that many of the benefits of the census don’t depend on absolute numbers but on relative numbers. This is self-evident in true of how many seats a state gets in Congress — count only 90 percent of every state, and each state would still get the right number of congressional seats. It is also true of many social benefits. Transportation and education grants, for example, are allocated proportionately. Areas and groups that are differentially undercounted are penalized, as the grant funds go to those places more fully counted. The only practical way to reduce the differential is to count 100 percent of everyone everywhere.
The consequences of not reaching that goal are substantial. The Veteran’s Administration wants to put a new hospital where it can serve elderly veterans. To do so, it needs measures of age and of veteran status that are accurate. A significant undercount puts the hospital in the wrong town. A poor-quality census means policies that miss their mark.
HF: Undercounts and overcounts seem technical, but they have led to big partisan fights. Can you explain what happened?
KP: The biggest controversy emerged in 2000. It was a controversy over whether we could sample the non-respondents. When we sent the forms out, 65 to 70 percent of the population responded automatically, but then you had to find the other 35 percent by knocking on doors or other procedures to ensure they were included. The Census Bureau knows that when you get down to the last hard-to-find groups — the last 4 to 5 percent of the population, who perhaps don’t have phones, or don’t speak English as their first language, or don’t trust the government — it was actually more accurate to sample the neighborhoods where the hard-to-count cluster than to try to find everyone.
At a certain point, you run out of money and time. It is better to sample, use your best staff to ensure that everyone in the sample is found, and use the sample results to estimate. When I was challenged about this — by critics of sampling — I sometimes replied: “Well, next time you go in for a blood test, tell them to take it all out.”
But sampling was ruled impermissible by the Supreme Court, at least for the apportionment count. Of course, the government does make extensive use of sampling in its surveys — such as the American Community Survey noted above, but also, heavily, in employment, economic activity, agriculture, health and more.
Sampling in 2000 became a partisan issue — Democrats favored it and Republicans were opposed. I was often asked if this was “political interference” in the census. The Constitution declares that Congress shall determine how the census will be conducted. It’s not political interference for the Congress to declare that it will tolerate or not tolerate a certain kind of procedure, in my judgment. What does represent political interference is to design a census procedure with the intent to gain partisan advantage. Under those circumstances, the director, I believe, is obligated to argue back, refusing to compromise census accuracy in pursuit of partisan goals, and to resign if unsuccessful in that effort.
HF: What else depends on the census?
KP: The decadal census has a critical role in guaranteeing the accuracy of all reputable sample surveys, whether conducted by the government or by a private research firm. Sample surveys use the census to adjust numbers to ensure that population characteristics — gender, age, race — match the census counts. When the census reports the age-structure of the population, every sample survey will check to see if its sample is in line.
If the census undercounts, overcounts, or counts people in the wrong place, these errors are reproduced in all sample surveys. Unemployment rates, health conditions, educational attainment are just a few of the statistics that can be affected; they are inaccurate to the extent that the decadal census is inaccurate. If the census is wrong, there is no other standard available. In this critical sense the census is basic to the entire information system that the country relies on.
In addition, we use the same census questions and framework from one census to another. This is what allows you to create good trend lines, which tell you what is happening across time. Is employment going up or down? Are school dropout rates going up or down? All of those numbers trace their trend lines back to the frame that the initial census puts in place for the country.
HF: What will happen if the census is carried out poorly?
KP: If we don’t advertise the census, if we don’t test the census, if we don’t use all the ammunition that we have to create a healthy census that people want to cooperate with voluntarily, the undercount will go back up. It has been steadily dropping since 1940, as we got better and better, and in 2000 and 2010 it reached all-time lows because funding allowed the Census Bureau, for the first time, to mount a sophisticated and professional advertising campaign.
The budget constraints and leadership vacuum puts this steady improvement at risk. At this time in its preparation phase, it is much more limited than, for example, in the two previous censuses. I am frequently asked: Is too late to have a good census in 2020? It is not too late, if Congress wants to fund it, and if Secretary Ross introduces strong leadership, it is not too late to recoup. But the clock is ticking. The deadlines are relentless.
If it’s just not a very good census compared to what it ought to have been, we can still use the numbers, though there will be problems at the edges. One state may end up getting the last seat that should have gone to another, and there may be Supreme Court rulings. There will be a lot of angry mayors, saying that the count of school kids was wrong, and needs to be redone.
It will be particularly damaging if the Census Bureau is unfairly blamed for a “failed” census (as it was, unfairly, in 1990). I worry about the crises of confidence, if the administration concludes that the Census Bureau is incompetent and this view spreads more generally to the federal statistical system. This will tempt some to say, “Well, let’s give the census to the private sector.”
That would be a mistake. The census is a public good, not a source of private profit. Moreover, the census and the entire federal statistical system understands and practices quality control, protects privacy, understands the importance of trend lines, creates public use files, and partners with thousands of groups that share responsibility for a good census.
This article is one in a series supported by the MacArthur Foundation Research Network on Opening Governance that seeks to work collaboratively to increase our understanding of how to design more effective and legitimate democratic institutions using new technologies and new methods. Neither the MacArthur Foundation nor the Network is responsible for the article’s specific content. Other posts in the series can be found here.