This week, I’ll be blogging about my recent Alabama Law Review article, Do Faith-Based Prisons Work? Here’s the abstract:

This Article examines everything we know about the effectiveness of faith-based prisons, which is not very much.

Most studies cannot be taken seriously because they are tainted by the “self-selection problem.” It is hard to determine the effect of faith-based prison programs because they are voluntary, and volunteers are more likely to be motivated to change and are therefore already less likely to commit infractions or be re-arrested. This problem is the same one that education researchers have struggled with in determining whether private schools are better than public schools.

The only credible studies done so far compare participants with non-participants who volunteered for the program but were rejected. Some studies in this category find no effect, but some do find a modest effect. But even those that find an effect are subject to additional critiques: for instance, participants may have benefited from being exposed to treatment resources that non-participants were denied.

Thus, based on current research, there is no strong reason to believe that faith-based prisons work. However, there is also no strong reason to believe that they do not work. I conclude with thoughts on how faith-based prison programs might be improved, and offer a strategy that would allow such experimentation to proceed consistent with the Constitution.

Here’s the Introduction:

There are five things one should know about faith-based prisons.

  • There are a lot of them out there. As of 2005, 19 states and the federal government had some sort of residential faith-based program, aimed at rehabilitating participating inmates by teaching them subjects like “ethical decision-making, anger management, victim restitution,” and substance abuse in conjunction with religious principles.
  • One of them—the InnerChange Freedom Initiative program in Iowa—was struck down on Establishment Clause grounds in 2006, but various faith-based prison programs still exist, including InnerChange programs in other states. InnerChange programs, which are explicitly motivated by Christian and Biblical principles, are probably more vulnerable to constitutional challenges; programs that are more interfaith and have less explicitly religious content, like Florida’s Faith- and Character-Based Institutions or the federal Life Connections Program, are probably less so.
  • Faith-based prisons continue to be promoted as promising avenues for reform, chiefly on the grounds that they improve prison discipline and reduce recidivism.
  • However, most of the empirical studies of the effectiveness of faith-based prisons have serious methodological problems and, to the extent they find any positive effect of faith-based prisons, can’t be taken at face value.
  • Those few empirical studies that approach methodological validity either fail to show that faith-based prisons reduce recidivism, or provide weak evidence in favor of them.

In what follows, I explain and critically evaluate the empirical studies of the effectiveness of faith-based prisons. The reader who gets through this Article will know everything that we currently know about whether they “work,” by which I mean, chiefly, whether they reduce in-prison infractions or some measure of post-release recidivism, such as time to re-arrest, probability of re-arrest, or probability of reconviction.

As the summary above indicates, we don’t know much about the effectiveness of faith-based prisons. This is a shame, because the empirics of faith-based prisons are important, both to the legal community and to policymakers generally.

First, we should care about the empirics because faith-based prisons and rehabilitative services are, and will continue to be, on the cutting edge of Establishment Clause litigation, and empirics matter in the law. One might think that whether a program works shouldn’t matter to whether it’s consistent with the Establishment Clause cases; but in fact, there are several areas in Establishment Clause doctrine that seem to allow targeted uses of statistical evidence.

In any event, regardless of whether empirics should matter in Establishment Clause cases, reading judicial opinions suggests that they do.

In Americans United for Separation of Church & State v. Prison Fellowship Ministries, which invalidated Iowa’s contract with the InnerChange Freedom Initiative, District Judge Robert Pratt, immediately before launching into his Establishment Clause analysis, concluded his description of the faith-based program with the following complaint:

More significant, however, [than the warden’s personal testimony about the program’s beneficial in-prison effect] is the lack of evidence presented by the Defendants about the effect of InnerChange on recidivism. Aside from anecdotes, the Defendants offered no definitive study about the actual effects the InnerChange program has on recidivism rates. [The warden’s] predecessor . . . communicated his desire early on in the initial RFP process that accountability for the program be included in the contractual agreement between the parties. Specifically, he requested “at least annual program evaluations to include, but not limited to, re-incarceration rates and other measurable outcomes.” But, in fact, there was no information presented at trial about whether InnerChange participants are more or less prone to recidivism than other inmates.

Of course, finding empirical language in opinions isn’t definitive evidence that the empirics are relevant. Maybe judges who claim to care about the effectiveness of faith-based programs are just indulging in legally meaningless rhetoric designed to support a conclusion they already arrived at by strictly legal means. On the other hand, if judges discuss empirical data, chances are that they believe the extra rhetorical force is useful in persuading someone and thus making their opinion more influential and less subject to reversal.

Moreover, if we put our Legal Realist hats on, maybe it’s the judges’ views of effectiveness that are driving their legal conclusions (at least for some judges, who aren’t in favor of or against faith-based prisons on ideological grounds). At the very least, if judges believe that a program is effective, they may think it’s a shame if the program were to be found unconstitutional and might therefore be extra careful in their legal analysis to avoid striking it down. (If we disagree with their conclusion, we might replace the word “careful” in the previous sentence with the word “imaginative.”) Conversely, if they believe a program is ineffective, they may feel no particular pressure to uphold it.

Perhaps a better piece of evidence that the empirics matter is that lawyers spend time discussing empirical studies in their briefs. Perhaps in response to Judge Pratt’s concern, when the case came up to the Eighth Circuit, the Alliance Defense Fund and others submitted an amicus brief arguing that “InnerChange’s faith-based rehabilitative prison programs are proven to reduce recividism.” (Unfortunately, of the two studies cited favorably in their brief, one has serious methodological problems, while the other, properly interpreted, shows no beneficial effect of the program.)

Empirical data also seems important in other Establishment Clause contexts. In Zelman v. Simmons-Harris (2002), which upheld an Ohio program that allowed publicly funded vouchers to be redeemed at religious schools, Justice Souter’s dissent cited statistics on the academic performance of schools to judge whether the parents’ choices were “genuine.” Justice O’Connor’s concurrence similarly used performance statistics to rebut Justice Souter’s argument.

Justice Thomas, for his part, noted that “the success of religious and private schools is in the end beside the point, because the State has a constitutional right to experiment with a variety of different programs to promote educational opportunity.” But that didn’t stop him from marshalling statistics to argue that “[r]eligious schools, like other private schools, achieve far better educational results than their public counterparts.” “That Ohio’s program includes successful schools,” Justice Thomas wrote, “simply indicates that such reform can in fact provide improved education to underprivileged urban children.” Here, too, seasoned appellate litigators spent several pages of their Supreme Court brief discussing the history and performance of voucher programs—and this was in a party’s argument, not just an amicus brief—though this discussion was ostensibly not “to convince the Court that parental choice is proper public policy.”

But perhaps more importantly, we should care about the empirics because, whether or not they should matter in the law, they obviously should matter in policy. If faith-based prisons don’t reduce recidivism, the case for their funding is correspondingly weakened (though they may still be supported by other arguments). If they do reduce recidivism, or if they have other policy advantages, then even if judges are wrong to stretch the law to find them constitutional, we aren’t wrong to try to find ways to allow them to function constitutionally. After all, even the non-religious have an interest in the rehabilitation of prisoners, and if religion can play a positive role in rehabilitation, this may be good news not only to the irreligious but even to those who are hostile to religion.

It is thus unfortunate that the legal literature hasn’t done a good job evaluating the empirical evidence on faith-based prisons. The law reviews are devoid of any comprehensive, critical discussion of the existing studies. Most legal articles on the subject simply choose not to bother with empirical studies. Some legal articles do address the empirical evidence, but their discussions are generally quite cursory.

One article supporting faith-based prisons spends a mere paragraph reciting, on the one hand, the claim that such units “show promise in their potential to reduce recidivism” and citing, on the other hand, methodological “concerns.” Another article, attacking faith-based prisons, limits its empirical analysis to critiquing the methodology of a single study. A third article argues more broadly that “[t]here is no scientific evidence . . . that faith-based rehabilitation programs are in fact effective at reducing recidivism,” but the discussion is almost entirely limited to a single study and review articles that likewise only discuss an incomplete subset of the studies. The most comprehensive law-review treatment of the empirical studies discusses five different empirical studies of faith-based prison programs, but it accepts their positive conclusions largely uncritically, whereas only three of these studies approach methodological soundness. Moreover, of those three, two, properly interpreted, show no effect, and the third shows, at most, weak effects.

Outside of the legal literature, a few review articles do take a broader approach. Some even single out which articles they believe are methodologically more valid than others. But, unfortunately for other scholars, these generally have no in-depth discussion of the studies being reviewed, nor do they discuss why the studies identified as superior really are superior. The rest of us are thus left to either take them at their word (I myself disagree with some of their assessments) or track down the studies (many of which are hard to find) and read them ourselves—a daunting task for those without empirical training.

*     *     *

This Article fills the void. More specifically, this Article makes three distinct contributions:

First, I provide a detailed critical discussion of (to my knowledge) every existing empirical study on the effectiveness of faith-based prison programs.

The word “every” in the last sentence masks a few critical decisions. In making such a survey, I have chosen to include certain kinds of studies and exclude others. Because the relevant legal issues center around “immersion”-style faith-based prison units “that seek to immerse prisoners in an almost monastic or total experience of religiously based living,” I exclude studies that explore more general issues like the effect of “religiosity” on adjustment to prison life, in-prison infractions, or recidivism. The measures of religiosity, in these studies, are either general measures of how religious an inmate feels or how many times an inmate attends religious services in prison. These studies may be valuable for some purposes, but they don’t help in evaluating immersion-style faith-based units, since ordinary prison worship services are both widespread and uncontroversial.

I instead focus on studies of the effectiveness of specific faith-based interventions on variables of interest like the likelihood of recidivism. I also include studies of religious after-care for released inmates, even though these aren’t technically in-prison programs.

Second, I provide a detailed discussion of the methodological issues involved in evaluating faith-based prisons generally. In statistics, methodology is everything; it’s a shame that the legal community, which often relies on these empirical studies, isn’t as sophisticated as it could be at telling valid studies apart from invalid ones.

Roughly speaking, the studies fall into four categories. Three of them—naïve comparisons of participants to non-participants, comparisons with some controls, and matching based on propensity scores—aren’t credible because they don’t account for what is, in my view, the most serious obstacle to effective assessment: the “self-selection problem.” Inmates who are motivated enough to choose to participate in a rehabilitative program are already less likely to reoffend. So any study that compares voluntary participants and voluntary non-participants may just be picking up the effect of being a good person, not the effect of the program itself. (Some of these studies are subject to even further sources of bias. For instance, in addition to self-selection in the decision whether and how intensively to participate, there can be selection by the program staff in the decision of whom to admit or whom to kick out, as well as “success bias” in the consideration only of those who completed the program without dropping out.)

In my view, the only credible studies so far fall into a fourth category—those that compare (voluntary) participants in faith-based programs with people who volunteered for the program but were rejected. And even these studies are subject to the “resources problem”: They compare participation in the program either with the alternative of no program at all or with the “business as usual” alternative of whatever other programs happened to be available, rather than with participation in a comparably funded secular program. Thus, even if a religious program is better than nothing at all, it could be because of the greater access to treatment resources (for instance, mentors and counselors) and not because of the religious content of the program.

Third, I set the empirical debate on faith-based prisons side by side with a parallel empirical debate: whether private schools are better than public schools. One striking aspect of the faith-based prisons research is how much it looks like the private school research. (Some of this research studies not private schools as such, but Catholic schools, since most private schools are religious, and Catholic schools are a homogeneous enough group to be susceptible to generalization.) Both faith-based prisons and private schools are subject to self-selection—any naïve comparison between, on the one hand, secular prisons or public schools and, on the other hand, faith-based prisons or private/Catholic schools is subject to the critique that the private or religious options have to be affirmatively chosen, and people who are motivated enough (or who have motivated enough parents) to make that choice are already more likely to be high-achieving students or low-recidivism inmates.

While sophisticated researchers in both areas are aware of the self-selection problem, the education literature has addressed it far more conscientiously and has progressed further than the prisons literature in advancing our empirical knowledge. Perhaps prison researchers could learn something from school researchers’ example.

*     *     *

In the end, this Article has bad news and good news.

The bad news, as I’ve said above, is that most studies are low-quality, and the results of the higher-quality studies aren’t promising. There seems to be little empirical reason to believe that faith-based prisons work.

The good news is that there’s also no proof that they don’t work. The absence of statistically valid or statistically significant findings isn’t the same as the presence of negative findings. And while the self-selection problem is real and important, the resources problem may not even be a problem at all: maybe the “zero alternative” or the “business as usual” alternatives really are proper empirical baselines, since they reflect both reality and, perhaps, political feasibility. So the picture isn’t uniformly bleak: there are some programs that seem to show some statistically significant effects, even if they are weak and even if we’re not sure how well they compare to the hypothetical effects of a hypothetical, comparably funded, secular program.

Perhaps future research will shed light on these questions. In the meantime, clearly some groups want to have such prisons, some inmates want to attend them, and they probably do little if any harm. If some programs don’t work, this is an indication to future practitioners that something needs to be changed; if some programs work, maybe they can be replicated elsewhere. Better results won’t emerge unless they’re allowed to emerge by a process of experimentation. As I’ve suggested above, it would be a shame if this process is cut off for constitutional reasons, provided there is a constitutionally valid way for them to proceed.

At the end of this Article, I suggest such a way. Faith-based prisons, as currently constituted, are probably unconstitutional under modern Establishment Clause doctrine. But they would become fully constitutional under a system of prison vouchers that would allow inmates to choose their own prisons, whether secular or religious. I develop this idea at length elsewhere, but the bottom line is that, despite the current weak evidence, some version of faith-based prisons may still work, and there is a way for that version to emerge consistent with the Constitution.

In tomorrow’s post, I’ll discuss some of the studies, starting with the least sophisticated ones.