Soon after he joined the Supreme Court, the late Justice Hugo L. Black moved into a stately brick home in Old Town Alexandria, where he lived for more than three decades until his death in 1971.

Black’s spacious garden and Federal-style home, where he drafted opinions, hosted presidents and played tennis, are designated a historic landmark and the late justice protected the property as open space.

But preservationists in Alexandria lost their battle in court Thursday to block renovations planned by the current owners of the South Lee Street property after Virginia’s highest court said the project could go forward.

The Virginia Supreme Court unanimously affirmed a lower court finding that the Historic Alexandria Foundation did not have legal grounds to appeal after the City Council signed off on the project. The court acknowledged the foundation was established “to advocate for the preservation of Alexandria’s historic buildings, districts, and neighborhoods,” but said the group’s mission “does not give it standing to challenge the City Council’s decision.”

The property, built around 1800, is owned by Capital One co-founder Nigel Morris and his wife, Lori, who said through their lawyers Thursday that they are pleased with the ruling. Attorney Gifford Hampshire said in court filings that the planned renovations are “consistent with the historic character of the house.”

The foundation’s lawyer, state Sen. Chap Petersen (D-Fairfax City), called the decision a disappointment with implications for historic preservation throughout the state. The group had the backing of some of Black’s former law clerks, his granddaughter, biographer and state preservation organizations.

The foundation, he said, is “charged with preserving the character of Old Town” and owns nearby properties. “If they can’t have a say in what goes on, it makes it very hard to enforce” other laws, Petersen said. “This was a marquee property and an important case.”

In a statement, a city spokeswoman said the property “is unquestionably an architecturally and culturally significant resource in Alexandria” and city officials had worked to “ensure that the historic district regulations meant to protect such properties were fully met.”

Kelly Gilfillen, acting communications director, said the court ruling makes clear that the process for public input, including from the Historic Alexandria Foundation, is at the City Council level.

In 1969, Black voluntarily granted an “open space” easement on his property that imposes restrictions on development to protect the historic structures and surrounding land at the intersection of Lee and Franklin streets. Black’s widow sold the property in 1973. After several owners, the property was sold most recently in 2013 for $6.25 million, according to city real estate records, to Vowell LLC, owned by the Morrises.

The dispute began in 2018 when the couple applied for and received permits from the city’s architectural review board to demolish parts of the home and make additions and other changes. That decision, opposed by more than 100 neighbors, was appealed and upheld by the City Council in 2019.

The foundation went to court and objected to plans to get rid of a curved wall of the house and install three brick “pavilion” structures in the garden.

Black’s former law clerks also opposed the changes and recalled working on opinions in Black’s study and playing tennis with the justice at his home. Black was a senator from Alabama when he was nominated to the court by President Franklin D. Roosevelt in 1937.

Despite his past as a member of the Ku Klux Klan, Black became an advocate for civil liberties and was part of the Brown v. Board of Education decision ending legal racial segregation in schools. He wrote the landmark opinion that gave criminal defendants the right to an attorney.

“We have no doubt that, if he were here, he would urge this court to prevent destruction of his legacy in Alexandria from a decision taken in disregard of Virginia law,” former clerks John G. Kester, George L. Saunders and A.E. Dick Howard said in a filing to the court.

The owners noted in their filing that the open-space easement on the property was changed in 1973 to allow for tennis court maintenance. It allows for “structural changes, alterations, additions or improvements” to the “manor house” with “prior written approval” from the Virginia Department of Historic Resources, according to the filing from the owners’ lawyer.

The department determined that the renovation complied with the easement and the city also gave its blessing to the plans.

But the law clerks said in their filing that if the house is “marred by the demolition and modern additions” it would no longer qualify as one of the foundation’s showcase authentic historic homes.

“A Virginia historic resource, once altered or destroyed, is gone forever,” they wrote.

For its part, the Virginia Supreme Court ruled narrowly on the issue of whether the foundation had a legal right to bring the case. The court’s unsigned order said the foundation had not suffered a specific “harm that differed from that suffered by the public in general.”

“Assuming that the City Council’s decision and the planned renovation of the property would indeed result in the harm alleged by the Foundation,” it said, “the resulting harm would be shared by the public generally.”

Magda Jean-Louis contributed to this report.