Peter Zeidenberg is a lawyer with DLA Piper. He was a federal prosecutor from 1991 to 2007 and spent 11 years at the U.S. Attorney’s Office for the District of Columbia.

When I was an assistant U.S. attorney in the District of Columbia, a common practice of narcotics officers was to walk the aisles of Amtrak trains when they stopped at Union Station and ask “suspicious” passengers for permission to search their bags. The officers were looking to intercept “mules” moving drugs to New York from Florida on behalf of major narcotics dealers. Although travelers had the legal right to deny these search requests, this was never explained to them beforehand, and many agreed to have their bags examined, often resulting in the seizure of large amounts of drugs.

My first federal trial involved a mule who had been stopped and searched in this manner. His backpack contained sizable quantities of cocaine and heroin. Either because of fear or, more likely, ignorance, this defendant was unable to assist in the prosecution of anyone higher up in the drug chain. This is hardly surprising. No savvy dealer tells a courier his true identity or address. And we were not about to release our defendant on the basis of a solemn promise that he would track down the supplier and then provide us with the information. So, with nothing to use as a bargaining chip, this unfortunate man went to trial, at which he was quickly convicted and sentenced to nearly 25 years in prison.

The decades-long imprisonment of such low-level drug dealers is not just. Fortunately, Attorney General Eric Holder has recently taken several modest but not insignificant steps that should help change this practice. First, in a break from precedent, Holder said last month that mandatory minimum sentences should be sought only for high-level or violent drug dealers. He has also endorsed modest changes proposed by the U.S. Sentencing Commission that would effectively lessen the sentences imposed for drug offenses by about one year.

It is not surprising that these proposed changes have drawn fire from certain prosecutors and “law and order” politicians. Eliminating the imposition of mandatory minimum sentences for nonviolent drug crimes would, without question, make the prosecution of drug trafficking cases more difficult. The how-to of taking down drug organizations is not particularly complicated: Arrest the low-level dealers and, after threatening them with the prospect of a lengthy prison sentence, obtain their cooperation and agreement to testify against their higher-ups. Long mandatory-minimum sentences are a crude but effective method of extracting this cooperation. So narcotics prosecutors’ resistance to the proposed reforms is to be expected.

But merely because it is often effective to threaten small-time drug dealers with decades in prison if they fail to cooperate does not make the practice just. Saudi Arabia imposes the death penalty for trafficking even small amounts of drugs and, by all reports, the kingdom has very low levels of illegal drug use. But few would suggest that such a draconian approach, even if effective, was just or moral. A balance must be struck between the crude effectiveness of the threat of a long prison sentence and the fairness of sentencing a low-level, nonviolent drug dealer — generally a user himself or herself — to decades behind bars. The cost of that imprisonment — not simply in taxpayer dollars, though that is immense — must also be considered in the context of sentences that violent felons typically receive.

Simply put, something is very wrong with a system in which low-level drug dealers who either cannot or will not cooperate with the government receive sentences comparable to, or longer than, those frequently meted out to armed robbers, rapists and even some murderers.

Will this change in approach make prosecutors’ jobs more difficult and permit some big-time drug dealers to go unpunished? Undoubtedly, and it would be naive to deny this reality. But the societal benefits of strong-arm prosecution tactics must be constantly calibrated against the costs. The recognition and consideration of these costs is long overdue.