The deepest secret in Washington is not what happened to the profits from PTL or the Iran arms sales, nor the name(s) of Gary Hart's most constant companion(s). The longest-running mystery is the origin of the term "Cave Dweller."

The slightly self-deprecating term is applied far too loosely these days, to designate Washingtonians who have linen tea napkins, belong to the Sulgrave/Cosmos/Metropolitan/Chevy Chase clubs, and remember at least one earlier revival of the Willard Hotel.

Douglas Woods Sprunt, with Judith Waldrop Frank and the Junior League of Washington, are responsible for "The City of Washington, An Illustrated History," published by Knopf 10 years ago and just issued as a paperback. Sprunt says she is not a Cave Dweller. "Though I was born here, my father was from Charlottesville and my mother from Richmond.

"I define as a Cave Dweller someone whose family has been here for several generations. It has to be your grandparents at least -- and some would think that's not strict enough."

No one could dispute the claim of Adlumia Sterrett (Mrs. Randall) Hagner, the District's vice regent for Mount Vernon, George Washington's estate. Her children are seventh-generation cave dwellers. Her family has lived on the Springland estate in northwest Washington for not quite 200 years.

"We used to call it the ant hill because so many of our aunts lived nearby," she says. The name "Adlumia" comes from her illustrious ancestor John Adlum, the great viticulturist who owned the land where the University of the District of Columbia now stands.

But her family goes back in these environs longer than that. "When I married Randall Hagner," Adlumia Hagner says, "we had some small dispute about whose family came to this area first. We finally agreed that both arrived pretty close upon each other in Maryland of the 1600s.

"I always heard the term 'Cave Dweller' from my parents and what not. I never understood it myself. It conjures up pulling someone by the hair while holding a club."

JeannineSmith (Mrs. Charles H.) Clark, a Smithsonian regent, traces her family back to Mary Jones in 1830. "She made and sold candy near Velati's famous candy store," says Clark. "Her son, William Jones, was a guard at the Capitol and listed in the little census book. They wore wonderful uniforms with epaulets. He built a house which still stands on the east side of 12th Street."

Jeannine Clark still has the 1878 award won by her ancestor George Jones, inscribed from "the Public Colored Schools of Wgtn and Gtown." He, however, left for Oklahoma, where he became a Republican National Committeeman, as did Jeannine Clark many years later.

TheClark grandchildren are also seventh-generation Cave Dwellers. "I think if you're at least a third generation you have a different perspective," says Clark. "I rode my bicycle in Crestwood as a child, but back then covenants wouldn't have allowed my parents to own a house in the neighborhood. Now we live in Crestwood. I'm sorry my father {John Archibald Smith} didn't live to see me appointed regent of the Smithsonian because he always took us there after Sunday School."

Jan (Mrs. Benjamin C.) Evans, whose grandfather Christian Heurich Sr. built the Heurich Mansion near Dupont Circle, has organized the Century Club, open to people who have had ancestors here at least 100 years. It's a club within the Columbia Historical Society, which has its home in Evans' family house. Brainard Warner ("I guess I'm president") says the group has 80 members. "About 20 of them have families who go back 150 years or so." His grandfather came here in 1862 from Pennsylvania.

As for the origin of the term Cave Dwellers, says Warner, "Well, I suppose the oldest inhabitants lived in caves. I think your family has to have lived here since at least before the turn of the century."

Decorator John Irelan, a fifth-generation C-D, says one way you can tell a certified Cave Dweller is by looking at the names on the tombstones in Oak Hill Cemetery. "Though you don't hear the names so often, anymore," he says.

The first published use of the term may be in a 1904 article in The Delineator magazine by the pseudonymous Marie Columbia. She wrote that after The Official Set and The Smart Set "there are the really, truly delightful old 'residenters' of Washington, made up mostly of well-born, well-bred families, long distinguished at the Capital for other qualities than riches or official position. They love to call themselves the 'Cave Dwellers,' and they open their doors only when and to whom they please, shutting them even in the face of the President of the United States if they see fit."

Today, with the vastly increasing invasion of lobbyists, corporation headquarters, and international Concorde jet-setters, the C-Ds may have to roll boulders to the entry of their Caves to defend the premises.

HOMECOMING OF THE YEAR: Montpelier, James Madison's family home, is giving a family reunion dinner June 28 for all collateral descendants of the fourth president. The National Trust for Historic Preservation is now giving Constitution Bicentennial events at the estate, at Montpelier Station, Va. 22957 (telephone 703-672-0006). Tours are offered daily from 10 a.m. to 4 p.m.