Influx of U.S. Inmates Slowing, Census Says

Indeed, said Visher, the prison population might even start to decline if it weren't for the high recidivism rate of those released: About half return to prison within three years.

And with a recent uptick in the crime rate, and increasing numbers of offenders being placed on probation, Beck said that the prison population may begin to significantly increase again.

Even if the prison population remains at its current level, the social and economic costs to the nation are enormous, Visher said.

She noted that the federal government and states are spending more than $65 billion per year on corrections alone. "We need to have a national conversation about how to transition this population into being productive," Visher said.

Just as worrisome is the persistent overrepresentation of blacks in prison. In 2006, blacks accounted for 12 percent of the general population but 40 percent of those in adult correctional institutions.

Hispanics are also overrepresented, but to a lesser extent. They made up 15 percent of the general population and 19 percent of the prison population.

By contrast, immigrants are underrepresented. Foreign-born inmates accounted for 9 percent of those incarcerated, compared with 13 percent of the total population.