In the wake of the terrible racist terrorist attack in Charleston, many are calling on South Carolina and other state governments to stop displaying the Confederate flag. They include a much-discussed Atlantic column by Ta-Nehisi Coates, and various politicians, such as 2012 GOP presidential candidate Mitt Romney.

It is long past time that state governments stopped displaying the Confederate flag, and otherwise honoring Confederate leaders. We should not honor people whose main claim to fame was waging a bloody war for the purpose of perpetuating and extending the evil institution of slavery. Some still deny that this was the motivation for Confederate secession. But overwhelming evidence proves otherwise. You don’t have to take my word for the proposition that slavery was at the root of the Confederate agenda, or even the word of John Stuart Mill. Take that of Confederate President Jefferson Davis, vice president Alexander Stephens (who called slavery “the cornerstone” of the Confederacy), and southern states’ own official statements outlining their reasons for seceding.

Despite claims that the Confederates’ true cause was “states’ rights,” they were in fact more than willing to sacrifice state autonomy to slavery whenever the two conflicted. For example, Confederate forces tried to coerce the states of Kentucky and Missouri into joining the Confederacy, even though majority opinion in both preferred to stay in the Union. The Confederate Constitution gave slaveowners the right of “sojourn” with their slave property in all states, effectively preventing state governments from excluding slavery from their territory.

The Confederacy cannot even be defended on on the traditional ground that its establishment represented the will of the people of the southern states – at least not if we remember that some 40% of its population was black. When you combine the African-American population (the vast majority of whom likely preferred to stay in the Union), with the substantial minority of southern whites who opposed secession, it becomes clear that secession did not enjoy majority support in the South. Majority approval doesn’t automatically justify a secession movement – particularly one with an agenda as awful as preserving slavery. But the Confederates lacked even that justification.

Unfortunately, flying the Confederate flag (as South Carolina still does) is not the only way that state and local governments continue to honor the Confederacy and its leaders. Throughout the South, there are still numerous schools, streets, and other institutions named after Jefferson Davis, Robert E. Lee (who, contrary to some popular mythology, was also a defender of slavery), and other Confederate bigwigs. This isn’t limited to conservative areas in the deep South. It is also true of places like liberal northern Virginia, where I live. You can’t drive more than a few miles here without seeing a street named after Davis, Lee, or Jeb Stuart.

One can try to defend the Confederates by invoking a kind of historical moral relativism: their support for slavery should be excused because it merely reflected the values of their time. But that actually lets them off the hook too easily. By 1861, many Americans – including some white southerners – recognized that slavery was wrong, and at odds with the Enlightenment ideals underlying the Constitution and the Declaration of Independence. A few southern whites were active opponents of slavery. Many more at least recognized that it wasn’t worth fighting a war to defend.

It is sometimes claimed that Robert E. Lee’s Virginia background in effect made his decision to support the Confederacy virtually inevitable. But some other Virginia-born officers, such as George Thomas (who became one of the top Union generals of the war), made the opposite choice. Perhaps the state government should honor those who made the right decision, rather than those who made a terribly wrong one. The Union side in the Civil War was far from ideal, and committed some injustices of its own. But its triumph was nonetheless far preferable to that of the Confederacy. As Frederick Douglass put it in 1871, it would be wrong to “remember with equal admiration… those who fought for slavery and those who fought for liberty and justice.”

Not all who wave the Confederate flag today do so because they approve of slavery and racism. Many, perhaps most, are simply trying to express regional pride, without carefully considering the flag’s history. Others have bought into the myth that Confederate secession had no real connection to slavery. But the flag’s historical association with slavery and racism cannot simply be ignored. Both during the Civil War era, and during other periods such as the Civil Rights Movement, its major function was as a symbol for political movements seeking to oppress blacks.

Taking down the Confederate flag and otherwise curbing official veneration of the Confederacy may not prevent racist violence of the kind we saw last week. Unlike participants in racist lynchings and mob violence a century ago, people like the perpetrator of the Charleston attack do not represent the mainstream values of their society. They are relatively marginal extremists who are unlikely to stop because most of society condemns them and their values. Nonetheless, ending state-sponsored honoring of Confederate leaders and symbols would be a valuable symbolic step.

UPDATE: I have made a few minor revisions to this post.