Statue of Confederate President Jefferson Davis, which was recently taken down in New Orleans.

Recent controversies over the removal of public Confederate monuments in New Orleans and Charlottesville, Virginia have rekindled debate over how we should deal with the historic legacy of the Confederacy. I agree with conservative columnist Jeff Jacoby, libertarian Ronald Bailey, and others who make the case for removal.

The issue comes down to this simple proposition: the government should not honor people whose principal claim to fame is that they fought a bloody war in defense of the evil institution of slavery. Ultimately, the case for removing Confederate monuments is the same as the case for removing the Confederate flag from public spaces of honor, and the case for renaming sites such as the Jefferson Davis Highway.

You don’t have to take my word for the centrality of slavery to the Confederate cause, or even the word of the overwhelming majority of Civil War historians. Take that of Confederate President Jefferson Davis himself, who unequivocally stated in 1861 that the cause of his state’s secession was that “she had heard proclaimed the theory that all men are created free and equal, and this made the basis of an attack upon her social institutions; and the sacred Declaration of Independence has been invoked to maintain the position of the equality of the races.” Or that of Davis’s vice president, Alexander Stephens, who famously avowed that “slavery . . . was the immediate cause of the late rupture and present revolution” and that protecting it was the “cornerstone” of the new Confederate government. Consider also the Southern states’ official statements outlining their reasons for secession, which focus on slavery far more than any other issue. Mississippi’s statement minced no words on the subject:

Our position is thoroughly identified with the institution of slavery– the greatest material interest of the world. Its labor supplies the product which constitutes by far the largest and most important portions of commerce of the earth…. These products have become necessities of the world, and a blow at slavery is a blow at commerce and civilization. That blow has been long aimed at the institution, and was at the point of reaching its consummation. There was no choice left us but submission to the mandates of abolition, or a dissolution of the Union…

The Confederacy cannot even be justified on the theory that the majority of the people in any state or region have a right to secede for any reason they want. As John Stuart Mill pointed out at the time, southern secession lacked majority support in any state, once you recognize that blacks count as people too, and were a part of the relevant population whose consent secessionists had an obligation to secure. African-Americans were some 40% of the population of the seceding states; it’s a safe bet that the overwhelming majority opposed secession. Between blacks and the substantial minority of southern whites who wanted to stay in the Union, it is likely that secessionists did not enjoy majority support in any state.

Despite the claims of some critics, removing Confederate monuments does not require any “whitewashing” of history. No one claims that we should erase the Confederacy and its leaders from the historical record. Far from it. We should certainly remember them and continue to study their history. We just should not honor them.

Perhaps some otherwise objectionable monuments should remain because they have great artistic or historical importance. Others can appropriately be displayed in museums and other facilities whose purpose is facilitating research and historical understanding, rather than bestowing honor. The same point applies to the use of Confederate flags in such settings as maps, boardgames, historical reenactments, and the like. The goal is not to literally remove all signs of the Confederacy, but to stop publicly honoring it and its leaders.

As Bailey and Jacoby note, it would be wrong to disturb Confederate cemeteries and other similar facilities whose purpose is to commemorate and lament the dead, rather than their deeds and the cause they fought for. Likewise, the government should not suppress the use of Confederate symbols and monuments by private organizations, including those that wrongheadedly seek to defend the Confederacy and its legacy. They are entitled to freedom of speech, no less than advocates of other causes. The state should not endorse such advocacy, but it also should not persecute it.

Some try to justify continuing to honor Confederates because we honor many other historical figures who committed various moral wrongs. For example, many of the Founding Fathers also owned slaves, just like many leading Confederates did. But the Founders deserve commemoration because their complicity in slavery was outweighed by other, more positive achievements, such as establishing the Constitution. By contrast, leading a war in defense of slavery was by far the most important historical legacy of Davis, Robert E. Lee, and other Confederate leaders. If not for secession and Civil War, few would remember them today.

The US is far from the only nation that struggles with such issues. Post-communist states in Eastern Europe and elsewhere have removed many communist monuments. Germany and Italy have removed monuments to the Nazis and Fascists. Communism, Nazism, fascism, and slavery should all remain part of our historical memory. But we should not honor the regimes and leaders who promoted these evils.