Peter Schuck is an emeritus professor at Yale Law School and the author of the provocative new book “Why Government Fails So Often, and How It Can Do Better.”  The book draws heavily on research in political science, economics, public administration, law, and related fields.  Schuck responded to several questions via email.  A lightly edited transcript follows.

Your book is an indictment of many, if not most, domestic programs at the federal level. How would we know if any particular government program has failed?

Chapter 2 defines what I mean by success, failure, and in-between. Essentially, I use a criterion of cost-effectiveness, arguing that no other criterion is workable and makes sense. I refine this criterion by discussing a number of its methodological constraints. These constraints include, for example, the difficulty of quantifying some costs and some benefits; the utilitarian assumptions on which such an analysis necessarily rests; the “precautionary principle;” data deficiencies; baseline issues; sensitivity analysis; confidence levels; and so forth. I emphasize at the end of this chapter that “policy success or failure is not simply in the eye of the beholder. . . .[G]ood policy assessment rests upon a number of well-established, relatively uncontroversial criteria that are certified in analytic methodology and routinely used in government practice. Although the application of these criteria to particular policies might be contested, it turns out that the results of such assessments are also remarkably consistent—and consistently negative” (pp. 62-3).

It might surprise some people that you consider Medicare a failure. Explain how you arrived at that assessment.

I do not count Medicare as a policy success despite its immense popularity and the universal access to vital health care for seniors that it affords. First, many national commissions have found for decades that because it reflects the large inefficiencies of the underlying health care system that it finances, Medicare is very cost-ineffective compared with plausible alternatives. Second, the most important policy implementers are three quarters of a million private providers whose strong professional commitments conflict with governmental priorities, and who need not accept Medicare patients at all and are increasingly refusing to do so. Third, and related, it made almost $48 billion in improper payments in 2010, far more than any other public program (and even more since then). Fourth, demographic trends and health care cost increases may render it insolvent by as soon as 2026.

You also document some successful government programs. One of them is food stamps. Why do you think of it as a success?

Like all programs directed at the alleviation of poverty, food stamps (now SNAP) has its problems, including fraud, erroneous payments, and exploding costs. But it is our most leading countercyclical entitlement program. Americans will not countenance children and poor people going hungry, especially during economic downturns. SNAP is well-targeted; roughly 45 percent of participants are children, nearly 75 percent are in families with children, and more than 25 percent are in households with seniors or people with disabilities. According to the Center for Budget and Policies Priorities, a leading advocate for the program, 92 percent of the expenditures go to eligible households, with the rest going to defray the federal share of administrative costs. Benefits can be quickly distributed to sudden disaster victims, and the benefit formula includes a substantial work incentive, which Congress recently strengthened.

When many people think about the failures of government programs, they think about mismanagement—bureaucratic incompetence, waste, fraud, and the like. How much would you say that mismanagement is to blame for government failure, compared to other factors?

It’s hard to say, of course, but I do show that fraud, waste, and abuse are endemic to federal programs, with many examples and numbers. My main argument, however, is that the deep structures of our policy system – perverse incentives, collective irrationality, poor information, systematic inflexibility, lack of credibility with necessary stakeholders, the superior speed, flexibility, and incentives of private markets, obstacles to implementation, the inherent limits of law as a policy instrument, and a mediocre and degraded bureaucracy – are the main causes of policy failure and mismanagement. Most mismanagement is a symptom and consequence of these deeper forces.

One theme of the book is how the American political system and its unique institutions affect the success or failure of policy. Perhaps the biggest villain is Congress. How does Congress make it difficult for government programs to succeed?

Let me count the ways, many of which are explained in the book. The short answer is that Congress causes or magnifies the deep structures I listed in my previous response. Much of this reflects our unique political system of separated powers, which enables Congress to intervene on every important policy decision in countless ways, and there is much to be said for such a system in terms of constraints on executive power, democratic accountability, responsiveness to affected interests, citizen penetration of bureaucracy, and much more – but it does have serious costs in terms of policy coherence and effectiveness.

Do you think that the prevailing solution to program failure is to eliminate programs or reduce the size of government, or to reform programs so that they have a better chance at success?

Both. Many programs should be eliminated, and others should be reformed in the interests of, as you say, a better chance of success.

You propose some possible reforms at the end of the book—acknowledging that they are incremental while also defending the value of incrementalism. Of these reforms, which do you think is most pressing?

The most pressing is the need for better information about (1) the likely effectiveness of programs before they are launched, and (2) their actual effectiveness in the past. Both of these require a substantial increase in the resources devoted to policy analysis and evaluation by technocratically competent and substantially independent agencies such as GAO, CBO, OMB, independent inspectors general, and outside assessment institutions.

You note early in the book that you are a “political independent” and “militant moderate” who nevertheless has voted for Democratic presidential candidates in every election except 1968. If you think that government programs too frequently fail, why do you tend to vote for the political party that is more committed to a significant role for government?

Big government is here to stay, and we need to make it work better, even as we try to prune it of programs that are ineffective or worse (which is part of what is necessary to make it work better). I also believe in a strong safety net for those who cannot help themselves sufficiently through market employment. I concluded that the Democratic presidential candidates were more likely than their Republican opponents to do both these things, especially as the GOP has become more extreme. But like many other voters, there of course are always other factors that affect my decision – for example, McCain’s selection of Sarah Palin was so irresponsible, given his age and her ignorance, that I preferred to take a chance on Obama, as inexperienced as he was in coalition-building and hard decisions.  Also, I tend to be much more liberal on most social issues than today’s Republicans.