The Arlington mansion, fronted by massive columns five feet around and a sweeping portico, sits high on a bluff overlooking the Potomac River with a commanding view of the nation’s capital. Two White men who lived in this storied home occupied a lofty perch as well: George Washington Parke Custis, the adopted son of George and Martha Washington, and General Robert E. Lee, commander of the Confederate Army during the Civil War.

And yet, as is so often true with American history, a scratch to the glossy surface soon reveals the African Americans who also spent their lives here. They were as remarkable as the families that had enslaved them.

There was Charles Syphax, who married the illegitimate Black daughter of Custis in the mansion’s parlor, an almost unheard-of privilege granted Maria Carter by her biological father. Some 40 years later, during the Civil War, Selina Norris Gray, the enslaved personal maid of Custis’s White daughter, who had married Lee, rescued family heirlooms from marauding Union soldiers.

Now, as monuments to the Confederacy fall and the names of those connected to the country’s racist history are removed from public buildings, a congressman from Virginia is proposing legislation to change the name of Arlington House, The Robert E. Lee Memorial to, simply, Arlington House.

Rep. Don Beyer (D-Va.), who has been reading Ibram X. Kendi’s “Stamped from the Beginning: The Definitive History of Racist Ideas in America,” said he is struck by how wrongheaded it is to memorialize Lee at the National Park Service site.

“It’s about recognizing, finally, that there’s been an enormous amount of systemic racism and that we have to own it and do our best to reverse it,” the Northern Virginia lawmaker said.

Descendants of Charles Syphax have been courting lawmakers for the past few months to make the change, said Syphax family historian Steve Hammond, who lives in Sterling, Va. The family‘s effort is motivated as much by a desire to accurately honor the full history of the property and the enslaved people who lived there as it is by any antipathy toward Lee.

Topping the list of the enslaved occupants of Arlington House estate are the Syphaxes, a prominent Black family whose achievements defied the odds. Its members include William Syphax, son of Charles and Maria Carter, who was appointed chairman of the D.C. Board of Trustees of Colored Public Schools. In 1870, he organized the college preparatory high school for Black students that would later become Dunbar High School, for decades one of the most prestigious Black schools in the United States.

“The fact that historically the site has Lee’s name attached to it automatically narrows how people think about the site,“ said Hammond, who is working with the National Park Service to re-envision how the home is presented when it is reopened this year after a restoration. "The stories of others who lived there get pushed to the background. My goal is to bring those stories to light.”

A complicated history

George Washington Parke Custis began building the mansion in 1803 on the 1,100-acre estate that his biological father had bequeathed him. Custis named the property Arlington after the Custis family homestead on Virginia’s Eastern Shore. Enslaved people were forced to build the imposing home, which was the first example of Greek Revival architecture in the United States. The mansion honored Custis’s adoptive parents — George Washington and Custis’s grandmother, Martha, by her first marriage — and housed an impressive collection of Washington family artifacts.

Like many prominent White men of his time, Custis had White children born through marriage and Black offspring born of rape. Maria Carter was the daughter of Custis and an enslaved woman named Arianna Carter. In 1821, Custis allowed Maria Carter to marry Syphax, a Mount Vernon enslaved man he inherited from his grandmother, in the mansion’s parlor. (It was a religious ceremony only; enslaved people could not legally marry.) Ten years later, in 1831, Mary Anna Custis married Robert E. Lee there, too, in the house where they raised their seven children.

Hammond surmises that, despite being enslaved, Maria Carter had a relatively tolerable existence inside the mansion as the personal maid of her younger White half-sister Mary Anna. Charles, who supervised the mansion’s dining room, and Maria Syphax had 10 children. Custis sold Maria and her first two children to the Quaker owner of an apothecary shop, who freed them, while her husband remained enslaved, albeit while living with his family, Hammond said.

Custis gave his Black daughter 17 acres of land on the Arlington estate, Hammond said. Maria Syphax’s “white cottage was surrounded by tall trees and pleasant stretches of grassland and the place was beautiful as well as homelike,” according to the Arlington Historical Society.

After the outbreak of the Civil War in 1861, Mary Custis Lee departed Arlington House to join her husband, leaving the mansion and many George Washington family heirlooms in Selina Norris Gray’s care. Union soldiers came to occupy the home that year, and Gray noticed that some relics had gone missing. She reported the stolen objects to Union Army Gen. Irvin McDowell, who secured the house from thieves who then shipped the remaining artifacts to the Patent Office for safekeeping, according to the National Park Service. In later decades, Gray’s children shared key details about the layout and furnishings of Arlington House that guided preservationists in restoring the home.

Why did Gray help rescue cherished artifacts from the family that enslaved her? Why would she care? Hammond suggests that as the personal maid to Mary Custis Lee, she felt a special connection to the first U.S. president. Also, he notes, the sprawling estate was Gray’s home, too, even if it amounted to one room for her husband, Thornton, and eight children in the slave quarters.

By the end of the Civil War, the Union dead were quickly filling up nearby military cemeteries so the U.S. Army began to bury them on the vast acreage surrounding Arlington House. In 1864, Arlington National Cemetery came into being, in part to dissuade Lee from ever trying to return.

The government first restored the house to the period of George Washington Parke Custis’s ownership, then in 1955 a joint resolution of Congress designated it the Custis-Lee Mansion. In 1972, Congress passed a law that gave the mansion yet another name: Arlington House, The Robert E. Lee Memorial.

Jump forward half a century and here we are: reckoning with our racist past as a nation. The Fairfax County School Board has voted to rename Robert E. Lee High School after the late congressman and civil rights giant John Lewis. Further south, in Richmond, the state is fighting in court to remove a Lee statue from Monument Avenue, as lawmakers in Congress push to move a statue of the Confederate general from the U.S. Capitol.

The Syphax family has been recruiting other lawmakers to join with Beyer to change the name.

Craig Syphax said that the mansion’s current name greatly overstates Lee’s role in the site’s history, which is primarily that of the Washington family and the generations of enslaved people who lived there.

Some Syphax ancestors are buried on the property. Sometimes, Craig Syphax said, when he visits the estate he senses their spirits. “I can feel something brush by me or touch me or a wind that comes through and no one knows where it came from.”

As he and Syphax work with the National Park Service to re-present the property, Hammond said he wants to bring his descendants and others who were enslaved on the property to life for visitors.

The family includes many leaders in the Arlington community and beyond. William’s brother, John, was born free and held several elective offices, including supervisor of the Arlington Magisterial District, delegate to the General Assembly and justice of the peace. Other family members include the legendary Howard University surgeon Burke “Mickey” Syphax, the late congressman Julian Dixon (D-Calif.) and activist-entrepreneur Tracey Syphax. The Syphaxes also fought in four wars, Hammond said, including three surviving World War II veterans now in their 90s.

Despite lobbying to remove Lee’s name from the house, Hammond confesses a desire embodied in the words of Martin Luther King Jr. “I have a dream that one day … the sons of former slaves and the sons of former slave owners will be able to sit down together at the table of brotherhood.”

He would like the Syphaxes to sit down with the descendants of Robert E. Lee, whose wife was the half sister of Maria Carter Syphax. At minimum they could acknowledge and explore the past, he said.

"But I believe that when you consider where our country is, it would be an extremely powerful statement for these two families to come together and talk about our past … and heal.”

An earlier version of this story misstated the number of surviving World War II veterans in the Syphax family. This version has been updated.

More from Retropolis: