SCANDAL PERMEATES the National Portrait Gallery's new exhibit on Henry Adams and his circle of friends and lovers, whose secret lives belied their public posture as eminent Victorians.

The inner ring of Adams's Washington circle was known as the Five of Hearts. It embraced Adams and Marion Hooper "Clover" Adams, his talented but tragically father-fixated wife; John Hay, private secretary to Abraham Lincoln and secretary of state to two other presidents; and Clara Stone Hay, an Ohio heiress and the only one of the group who practiced what she preached. The fifth and most notorious "Heart" was Clarence King, first chief of the U.S. Geological Survey, whose ambition was exceeded only by his libido and who lived much of his life as a lie.

The outer ring of the inner circle included Adams's lover, "Lizzie" Cameron, wife of Sen. J. Donald Cameron, an alcoholic Pennsylvanian who introduced as his own resolutions written by Adams; Hay's lover, "Nannie" Lodge, wife of Sen. Henry Cabot Lodge; and King's secret common-law wife, Ada Copeland Todd, a black woman who bore him five children but never knew her husband's real name or occupation until after he died. King, who ran through several fortunes -- not all of them his own -- left his widow destitute, but Hay, acting anonymously, created a modest life trust for her.

The rings of the outer circles included just about every notable figure in turn-of-the-century society, politics and the arts. Among those handsomely represented in the exhibit are such authors as Mark Twain, Henry James, Francis Parkman, William Dean Howells, Edith Wharton and Bret Harte.

Some of the finest portraits are camera studies made by Clover Adams, known as Washington's most brilliant hostess but frustrated by her role as an auxiliary to her famous husband and the lack of an outlet for her artistic talents (most of her photographs are being shown here in public for the first time). She committed suicide at 42 by swallowing some of the potassium cyanide she kept for retouching her photographs.

Scandal aside, this elegant exhibit is full of artistic and historic resonances. Worth a visit itself is the juxtaposition of two portraits by John Singer Sargent: his 1885 study of Robert Louis Stevenson and his wife, which is one of Sargent's most daring and successful compositions, and his 1903 portrait of Teddy Roosevelt, which may be the shallowest and shabbiest canvas Sargent ever executed.

A poignant footnote to history appears in the caption to robber baron Andrew Carnegie's portrait. He offered to buy independence for the people of the Philippines by paying back the $20 million the United States gave Spain for formal title to the islands after the Spanish-American War. The United States was buying nothing but trouble in pursuing imperial ambitions, Carnegie said, vainly urging Secretary of State Hay to persuade President McKinley to accept his offer. Perhaps in another century or so we'll be able to calculate the ultimate cost in blood and treasure of the American pursuit of its "Manifest Destiny."

The exhibit grew out of Connecticut writer Patricia O'Toole's shrewd and delightful history, "The Five of Hearts," being published simultaneously by Clarkson Potter (459 pp., $25). O'Toole's research turned up many of the portraits in the show, of which she is guest curator; for that public service alone she's entitled to a bestseller.