"No question about it, violent crime will go up," New York police commissioner Ray Kelly declared on Meet the Press on Sunday, when asked if a recent ruling striking down the city's "stop and frisk" policy would cost lives. "What we’re doing -- and what we’re trying to do -- is save lives," he added on This Week.

Actually, there are some questions about it. The research is clear: Stop and frisk is applied racially unevenly. But there's precious little evidence that it has worked to reduce crime. And then there's the question of whether it could actually be undermining effective policing by alienating the very communities it's meant to help.

First up: Smith and Purtell

There have been two studies to date evaluating the effectiveness of stop and frisk. The first, an unpublished paper by NYU's Dennis Smith and SUNY Albany's Robert Purtell, found "the strategy was effective city-wide for robbery, murder, burglary and motor vehicle theft." No city-wide impact was found for assault, rape, or grand larcery.

The homicide results were perhaps the most promising. "Stop and frisk appears to have been an effective city-wide strategy against murder resulting in a drop of -.0002 murders per thousand people for each increase of one stop per thousand," they find. But for other violent offenses, namely assault and rape, the results are nonexistent. Further, they find declining returns to scale on most crimes, which suggests that paring back stops could have relatively small costs even in regards to crimes where stop and frisk was found to work.

But that is, note, an unpublished paper. That means it hasn't withstood the scrutiny of peer review.

But then came Rosenfeld and Fornango

The second study (free copy here), by University of Missouri-St Louis's Richard Rosenfeld and Arizona State's Robert Fornango, throws cold water on even Smith and Purtell's modest positive findings on robbery and burglary. They find the stops "show few significant effects of several SQF [stop, question, and frisk] measures on precinct robbery and burglary rates." Unlike Smith and Purtell, this paper was both published and made it into a peer-reviewed journal.

It also corrects some issues in the Smith and Purtell paper. The earlier study failed to take into account factors that could effect both the rate of stops and the crime rate, and only looked at stop rates a month before crime rates. For example, it looked at the rate of stops in April in trying to explain the May crime rate. That isn't a very long lag, Rosenfeld and Fornango argue. And if you allow a longer lag, the effects vanish.

That critique is shared by SUNY Albany's Steven Messner and Florida State's Eric Baumer, who, in a paper commenting on Rosenfeld and Fornango, write, "it is clear that the study offered by Rosenfeld and Fornango is methodologically more comprehensive than its predecessor." Rosenfeld and Fornango's paper, they continue, is "a thoughtful, clearly written paper that goes substantially beyond existing empirical assessments of the efficacy of [stop and frisk]."

Legitimacy matters too

So at the end of the day, we have (a) Smith and Purtell, which finds encouraging results for some crimes but has methodological issues; (b) Rosenfeld and Fernango, which found no results, had a stronger methodology than Smith and Purtell, and passed peer review.

Meanwhile, the costs of the policy need to be accounted for, too. Stop and frisk is alienating the communities it targets. Perhaps the most famous -- and tragic -- example came in the late 1990s, when stop and frisk incidents ratcheted considerably and culminated in the death of Amadou Diallo, an innocent 22-year-old West African immigrant who was shot 41 times by NYPD officers as part of a stop. That spurred an investigation by the New York attorney general's office, then headed by Eliot Spitzer, into that policing program.

Such incidents have real costs. Columbia Law School's Jeffrey Fagan, Yale Law School Tracey Meares and NYU's Tom Tyler note that there's a huge research literature showing that perceptions of police legitimacy matter for crime rates -- in part because, for police to actually solve crimes, the community needs to want to work with them.

"If police are stopping lots and lots of young people who are innocent of any wrongdoing, and that produces tension and anger, and it happens to the same young people again and again, their inclination to cooperate with police in a subsequent investigation, even into their own victimization, could weaken substantially," Rosenfeld argues.

Is stop and frisk the only possible explanation for dropping crime rates?

There are plenty more plausible potential causes for New York's decline in crime, even relative to other parts of the country. Some involve policing tactics. Frank Zimring at Berkeley Law credits hot spot policing with much of the city's progress in his book, The City that Became SafeBut he's also said we don't know if stop and frisk works. Rosenfeld argues that the pent-up demand for development in New York — which the initial crime drop unleashed and which in turn reduced crime further, creating a virtuous circle — probably had an impact. My favorite explanation, of course, is lead poisoning, which was much lower in the mid-1990s — when today's criminals would have been born, and could have been poisoned by lead — in New York City than it was in other cities. Less poisoning means less aggression which, in turn, means less crime:

Deleading and hot spot policing are both proven, evidence-based crime reduction policies, while stop and frisk remains unproven. "Unproven," of course, doesn't mean "ineffective." David Weisburd, a criminologist at George Mason, argues that the program counts as "hot spots policing," an approach where police zero in on specific blocks or other precise geographic areas where crime is high. Hot spots policing has been proven effective in multiple randomized trials.

But Weisburd notes that even if the hot spots element of stop and frisk reduces crime, there are better ways to do it. For example, you could just have an officer positioned at the hot spot without having police stop people. "Why not develop a program to limit the use of stop and frisk to a limited group of people, and increase the use of high-legitimacy police tactics?" Weisburd says, citing the officer presence approach as an example of the latter.

Ultimately, Weisburd argues that this is the kind of question that you need a true experiment to resolve. "Let's say they did a randomized experiment, with 500 control blocks and 500 treatment blocks, where you measure the number of minorities who live on each block," Weisburd says. "They would have answers to the questions being asked." Until then, Kelly is operating on the basis of precious little evidence, and in the face of serious drawbacks to the approach he's chosen.