Why not measure prison performance?

February 24

Today I’m going to start blogging about my new article, Prison Accountability and Performance Measures, which is in the current issue of the Emory Law Journal. Here’s the abstract:

A few decades of comparative studies of public vs. private prison performance have failed to give a strong edge to either sector in terms of quality. That supposed market incentives haven’t delivered spectacular results is unsurprising, since, by and large, market incentives haven’t been allowed to work: outcomes are rarely measured and are even more rarely made the basis of compensation, and prison providers are rarely given substantial flexibility to experiment with alternative models.

This Article argues that performance measures should be implemented more widely in evaluating prisons. Implementing performance measures would advance our knowledge of which sector does a better job, facilitate a regime of competitive neutrality between the public and private sectors, promote greater clarity about the goals of prisons, and, perhaps most importantly, allow the use of performance-based contracts.

Performance measures and performance-based contracts have their critiques, for example: (1) the theoretical impossibility of knowing the proper prices, (2) the ways they would change the composition of the industry, for instance, by reducing public-interestedness or discouraging risk-averse providers, and (3) the potentially undesirable strategic behavior that would result, such as manipulation in the choice of goals, distortion of effort away from hard-to-measure dimensions or away from hard-to-serve inmates, or outright falsification of the numbers. I argue that these concerns are serious but aren’t so serious as to preclude substantial further experimentation.

Before I begin on the article, here are some quotes I like, which nicely illustrate the problem of input-based accountability. First, one from Charles Dickens’s Little Dorrit:

Here arises a feature of the Circumlocution Office, not previously mentioned in the present record. When that admirable Department got into trouble, and was, by some infuriated member of Parliament . . . attacked on the merits . . . as an Institution wholly abominable and Bedlamite; then the noble or right honourable [member] who represented it in the House, would smite that member and cleave him asunder, with a statement of the quantity of business (for the prevention of business) done by the Circumlocution Office. Then would that noble or right honourable [member] hold in his hand a paper containing a few figures, to which, with the permission of the House, he would entreat its attention. . . . Then would the noble or right honourable [member] perceive, sir, from this little document, which he thought might carry conviction even to the perversest mind . . . , that within the short compass of the last financial half-year, this much-maligned Department . . . had written and received fifteen thousand letters . . . , had made twenty-four thousand minutes . . . , and thirty-two thousand five hundred and seventeen memoranda . . . . [T]he sheets of foolscap paper it had devoted to the public service would pave the footways on both sides of Oxford Street from end to end, and leave nearly a quarter of a mile to spare for the park . . . ; while of tape—red tape—it had used enough to stretch, in graceful festoons, from Hyde Park Corner to the General Post Office. . . . No one . . . would [then] have the hardihood to hint that the more the Circumlocution Office did, the less was done, and that the greatest blessing it could confer on an unhappy public would be to do nothing.

Next, one from DOJ’s 2012 Environment & Natural Resources Division Annual Report:

The results obtained from ENRD’s civil and criminal cases in fiscal year 2012 alone were outstanding. We secured over $397 million in civil and stipulated penalties, cost recoveries, natural resource damages, and other civil monetary relief, including almost $133 million recovered for the Superfund. We obtained over $6.9 billion in corrective measures through court orders and settlements, which will go a long way toward protecting our air, water and other natural resources. We concluded 47 criminal cases against 83 defendants, obtaining nearly 21 years in confinement and over $38 million in criminal fines, restitution, community service funds and special assessments.

And now, here goes:

“Isn’t everything to be said on [private prisons] already in print?” asks Sharon Dolovich. She means the question to be merely rhetorical; and so do I. The comparative effectiveness debate, to the extent it’s relevant—and I think it is—has stalled, simply because the empirical literature, exhaustive as it is, is so bad. “The current weight of the evidence on prison privatization in the United States is so light that it defies interpretation,” write prison researcher Gerald Gaes and his coauthors. (The theory isn’t much better: the same authors characterize prison performance as a “theoretically bereft domain.”) To intelligently choose between public and private provision, we should at least know which sector costs less, but we don’t; and we should at least know which sector provides higher quality, but we don’t have a great sense of that either.

This seems puzzling: readers of the voluminous debate on private prisons can be forgiven for thinking that market incentives should make private prison firms either (1) cut wasteful expenditures and produce innovative services or (2) cut corners on essential inmate care and security and lead to a humanitarian disaster. Let’s focus on the positive claims for private prisons: if the private sector is so clearly superior, shouldn’t the difference hit us between the eyes?

On second thought, this isn’t so puzzling after all. The advantages of market provision are often said to be that, what with the rigidities and low-incentive structure of government agencies, private firms have greater incentive and greater flexibility to figure out how to achieve any desired level of quality. But this assumes that (1) particular levels of quality are desired or encouraged, and (2) private firms are given the flexibility to achieve these levels. It turns out that both of these assumptions are wrong.

Let’s take the quality problem first. Why not tally up the quality at a public prison, do the same at a comparable private prison, and compare the two quality measures? The trouble here is that—despite the scores of studies that have been produced purporting to measure quality differences—good performance measures are rarely used. As I document in Part I, this means that comparative quality studies are hard to interpret if one wants to know which sector is better. (This hasn’t prevented both partisans and detractors of private prisons from producing loosely reasoned pieces that oversell the findings of their favorite studies.)

It doesn’t have to be that way. Criminologists have produced no shortage of performance measures that are appropriate for evaluating prisons, using variables like in-prison violence, the quality of prison health care, the degree of crowding, and—which I think is immensely important—recidivism. The most important thing about a performance measure is that it measure performance, that is, outcomes. Inputs like money spent, guards hired, or programs offered are of quite limited value, since the whole point is to see whether the money spent is worthwhile, whether the guards hired are necessary, and whether the programs are effective. Outputs like the number of doctor visits or the number of graduates of rehabilitative programs—like the number of memos written by Dickens’s Circumlocution Office or the number of years of prison resulting from DOJ prosecutions—are also of limited value. Doctor visits might just be make-work; the rehabilitative programs may not actually be rehabilitative. (The Circumlocution Office, whose function is to prevent things from being done, has a zero or negative contribution to performance; and the prosecutions that maximize prison time aren’t necessarily the same as those that most improve the environment.) What we care about—prisoner health, decent conditions, actual rehabilitation—are the outcomes that we should actually measure, to the extent possible.

Why should we use performance measures? There are several reasons, which I’ll canvass in Part II of this Article.

First, it’s good just to know whether the public or private sector has higher quality, for instance in evaluating whether one’s state should outsource or insource a particular project, or whether it should be one of the nineteen states that don’t use private prisons at all. Naturally, many factors determine performance other than the quality of the management and the facilities: for instance, a prison can have better performance numbers because it was sent a better crop of people. But certainly having performance measures is better than useless.

Second, using performance measures would help to implement a regime of competitive neutrality, where the public and private sectors could bid against each other and individual projects could shuffle from one sector to another. Competitive neutrality might be better than an all-public or all-private regime, but to implement it properly, the auctions should be evenhanded, which means that proposed costs and proposed quality targets should be fairly comparable. Performance measures would allow a winning contractor to commit to deliver a particular level of performance, and would allow governments to levy the appropriate contractual fine if this level isn’t achieved (or grant the appropriate reward if the level is exceeded).

Third, it would help policymakers express what’s desirable in prisons. One would think that this had been done already; but prison contracts are written in input and output terms because this is largely how the industry works and thinks. Performance measures have been a byproduct of the debate over prison privatization: the different sides in the debate needed them to argue in favor of or against privatization; and the development of these measures has in turn spurred serious thinking about what prisons should accomplish, which has had accountability benefits for the public sector as well.

Perhaps most importantly, the use of performance measures would allow the spread of performance-based contracting, where—instead of levying a fine for not delivering a particular level of performance—one varies the contract fee continuously with the level of performance delivered. Once accountability is tied to actual performance—as is actually being done in the U.K.—giving prison providers the flexibility to choose how to do their job becomes more attractive.

Part III of this Article discusses critiques of using performance measures as part of a compensation scheme.

One concern is that the true social benefits of various aspects of performance are unknowable, either in principle or in practice, so that determining the proper prices will inevitably fail. Where a service is closely bound up with justice concerns, a focus on efficiency pricing may be inappropriate: it might demean the service or give insufficient weight to non-efficiency goals.

A second problem is that the use of performance measures will alter the composition of providers in the industry, in ways that are perhaps undesirable. One way this might happen is that, in the presence of monetary incentives, public-interested people may be less attracted to corrections. A different way performance measures can alter the composition of the industry is by increasing risk for providers. Providers can only control inputs, and the connection between inputs and outcomes is highly variable, because it depends on a great many variables, many of which are beyond the prison’s control—such as general social conditions or the underlying quality of the inmates. The relationship between any of these variables and outcomes is not very well known. One might care about the fairness of rewarding or penalizing providers based on factors beyond their control, though in an auction system, such windfalls will be canceled out by competitive bidding. More seriously, the riskiness might bias the set of available providers in favor of the largest and best-capitalized firms, and perhaps discourage experimentation with risky but promising techniques. This means that the sensitivity of price to outcomes might have to be limited, which might also limit the incentive effects.

A third problem is that providers may engage in undesirable strategic behavior. They might manipulate the performance goals so they are easy to meet. They might focus their effort on the measurable dimensions of performance and slight the unmeasurable ones. (For example, what are the true outcomes of the justice system? Some outcomes, like case backlogs, are measurable, but other important outcomes, like accuracy of adjudication, aren’t—and measuring one runs the risk of distorting the agency’s effort away from the unmeasured outcomes.) Similarly, providers will want to choose the easiest-to-treat populations (“creaming” or “cherry-picking”), and (given a population) fail to treat the hardest-to-treat members (“parking”). And, of course, any system based on particular numbers comes with the risk that someone might try to falsify the numbers.

The good news is that, for prisons, there’s hope that these concerns can be fairly addressed. At the very least, these concerns don’t seem so serious as to preclude far more experimentation than has been happening so far. We actually have access to reasonably good performance measures that reasonably cover the important dimensions of prison quality, none of which have to be limited to efficiency-based measures. These measures should be set by corrections departments, not by contractors. Riskiness can be addressed, at least in part, by only making part of the payment depend on performance. Social impact bonds have some promise in encouraging nonprofit-sector financing; in any event, the prison market is already highly concentrated, so there is currently no vast population of nonprofits and small companies to lose. Cherry-picking can be addressed by giving contractors no say in what inmates they’re given, and parking can be addressed, at least in part, by making monetary rewards depend on observable characteristics of the inmate (if, indeed, it’s a problem at all). Outright falsification of performance measures is a serious problem, which requires seriously investing in monitoring and ensuring robust disclosure regimes.

None of these are perfect fixes, but we don’t need perfection; we just need an improvement over the status quo.

In tomorrow’s post, I’ll discuss why it’s hard to figure out which costs less, public- or private-sector prisons.

Sasha Volokh lives in Atlanta with his wife and three kids, and is an associate professor at Emory Law School. He has written numerous articles and commentaries on law and economics, privatization, antitrust, prisons, constitutional law, regulation, torts, and legal history.
SECTION: {section=national, subsection=null}!!!
INITIAL commentConfig: {includereply=true, canvas_permalink_id=washpost.com/8bvh5zpd9k, allow_comments=true, commentmaxlength=2000, includeshare=true, display_comments=true, canvas_permalink_app_instance=m6yzjj840m, display_more=true, moderationrequired=false, includefeaturenotification=true, canvas_allcomments_id=washpost.com/km4ey0dajm, comments_period=14, defaultsort=reverseChronological, includevoteofftopic=false, allow_videos=false, includesorts=true, markerdisplay=post_commenter:Post Commenter|staff:Post Writer|top_commenter:Post Forum|top_local:Washingtologist|top_sports:SuperFan|fact_checker:Fact Checker|post_recommended:Post Recommended|world_watcher:World Watcher|cultuer_connoisseur:Culture Connoisseur|weather_watcher:Capital Weather Watcher|post_contributor:Post Contributor, childrenitemsperpage=3, includeheader=true, includeverifiedcommenters=true, defaulttab=all, includerecommend=true, includereport=true, maxitemstop=2, source=washpost.com, allow_photos=false, maxitems=7, display_ugc_photos=false, includepause=true, canvas_allcomments_app_instance=6634zxcgfd, includepermalink=false}!!!

UGC FROM ARTICLE: !!!

FINAL commentConfig: {includereply=true, canvas_permalink_id=washpost.com/8bvh5zpd9k, allow_comments=true, commentmaxlength=2000, includeshare=true, display_comments=true, canvas_permalink_app_instance=m6yzjj840m, display_more=true, moderationrequired=false, includefeaturenotification=true, canvas_allcomments_id=washpost.com/km4ey0dajm, comments_period=14, defaultsort=reverseChronological, includevoteofftopic=false, allow_videos=false, includesorts=true, markerdisplay=post_commenter:Post Commenter|staff:Post Writer|top_commenter:Post Forum|top_local:Washingtologist|top_sports:SuperFan|fact_checker:Fact Checker|post_recommended:Post Recommended|world_watcher:World Watcher|cultuer_connoisseur:Culture Connoisseur|weather_watcher:Capital Weather Watcher|post_contributor:Post Contributor, childrenitemsperpage=3, includeheader=true, includeverifiedcommenters=true, defaulttab=all, includerecommend=true, includereport=true, maxitemstop=2, source=washpost.com, allow_photos=false, maxitems=7, display_ugc_photos=false, includepause=true, canvas_allcomments_app_instance=6634zxcgfd, includepermalink=false}!!
Comments
SECTION: {section=national, subsection=null}!!!
INITIAL commentConfig: {includereply=true, canvas_permalink_id=washpost.com/8bvh5zpd9k, allow_comments=true, commentmaxlength=2000, includeshare=true, display_comments=true, canvas_permalink_app_instance=m6yzjj840m, display_more=true, moderationrequired=false, includefeaturenotification=true, canvas_allcomments_id=washpost.com/km4ey0dajm, comments_period=14, defaultsort=reverseChronological, includevoteofftopic=false, allow_videos=false, includesorts=true, markerdisplay=post_commenter:Post Commenter|staff:Post Writer|top_commenter:Post Forum|top_local:Washingtologist|top_sports:SuperFan|fact_checker:Fact Checker|post_recommended:Post Recommended|world_watcher:World Watcher|cultuer_connoisseur:Culture Connoisseur|weather_watcher:Capital Weather Watcher|post_contributor:Post Contributor, childrenitemsperpage=3, includeheader=true, includeverifiedcommenters=true, defaulttab=all, includerecommend=true, includereport=true, maxitemstop=2, source=washpost.com, allow_photos=false, maxitems=7, display_ugc_photos=false, includepause=true, canvas_allcomments_app_instance=6634zxcgfd, includepermalink=false}!!!

UGC FROM ARTICLE: !!!

FINAL commentConfig: {includereply=true, canvas_permalink_id=washpost.com/8bvh5zpd9k, allow_comments=true, commentmaxlength=2000, includeshare=true, display_comments=true, canvas_permalink_app_instance=m6yzjj840m, display_more=true, moderationrequired=false, includefeaturenotification=true, canvas_allcomments_id=washpost.com/km4ey0dajm, comments_period=14, defaultsort=reverseChronological, includevoteofftopic=false, allow_videos=false, includesorts=true, markerdisplay=post_commenter:Post Commenter|staff:Post Writer|top_commenter:Post Forum|top_local:Washingtologist|top_sports:SuperFan|fact_checker:Fact Checker|post_recommended:Post Recommended|world_watcher:World Watcher|cultuer_connoisseur:Culture Connoisseur|weather_watcher:Capital Weather Watcher|post_contributor:Post Contributor, childrenitemsperpage=3, includeheader=true, includeverifiedcommenters=true, defaulttab=all, includerecommend=true, includereport=true, maxitemstop=2, source=washpost.com, allow_photos=false, maxitems=7, display_ugc_photos=false, includepause=true, canvas_allcomments_app_instance=6634zxcgfd, includepermalink=false}!!
Most Read National
Next Story
Eugene Volokh · February 24