A visitor ushered into the senator's private office Friday in the Russell Building would have been hard pressed to identify its occupant by party affiliation: There were busts of Lincoln and Kennedy; pictures of his children with Presidents Eisenhower, Johnson and Ford, and of the senator with former colleagues Frank Church and Mike Mansfield, and pens commemorating passage of the 1964 Civil Rights Act and a 1976 bill limiting presidential power.

The biggest clue to his party tie had been covered inadvertently by a rumpled raincoat, tossed over a three-foot-tall teakwood elephant, a gift of the Cumberland, Md., Republican Club.

The coat's owner, Sen. Charles McC. Mathias, was down the hall in the ornate Senate Caucus Room, where before enough television cameras and applauding admirers to change the mind of a less determined politician, he announced that he would not seek a fourth term next year.

Mathias' sudden announcement, which he refused to reveal early to aides and colleagues, easily fit his image as a maverick. Throughout his career, he had defied labels. Despite his tweedy looks and upper-crust background, Mathias drove a 10-year-old station wagon through his campaigns and insisted he was a farmer at heart, a senatorial-looking figure who did not hesitate to stop on his way into work to walk by himself through the muddy fields of Resurrection City in 1968. And although he stubbornly clung to the Republican Party, more often than not he was allied with Democrats.

Some of those alliances, especially his lifelong friendship with the late NAACP lobbyist Clarence Mitchell, were manifested in Mathias' efforts on civil rights. He was considered crucial in getting the bill guaranteeing fair housing rights through the House in the 1960s, and he worked hard to establish a national holiday honoring slain civil rights leader Martin Luther King Jr.

Mathias also takes great pride in his environmental work, including legislation to study and clean up the Chesapeake Bay. In recent years, Mathias has become involved in pressuring the administration to work toward arms talks with the Soviet Union.

With Mathias' retirement, the Maryland Republican Party is losing its only candidate who ever carried the city of Baltimore in a statewide race, and its only member to be elected to the Senate from the state three times. In his absence, the Maryland GOP is without a popularly elected leader, adding to the national party's worry about retaining control of the Senate.

Not that the GOP has always had a vigorous advocate in Mathias. In his last campaign for reelection, in 1980, he went out of his way to disassociate himself from his party's presidential nominee. He declined to serve on a state Republican advisory committee for Ronald Reagan, snubbed a Reagan rally in Silver Spring and managed to campaign largely without identifying his political affiliation.

In his own campaign, Mathias staked out the traditional liberal positions on abortion, ERA and defense. He won the battle, trampling the Democratic nominee, the late state senator Edward T. Conroy -- whose views were clearly to Mathias' right -- by 2 to 1, but Mathias may have lost the war.

The Reagan landslide helped his party capture control of the Senate, but it did not require an elephant-like memory on the part of his conservative colleagues to remember that Mathias had refused to commit in advance to voting with his party if it meant giving chairmanships to conservative Republicans named Thurmond, Tower, Garn and Helms.

When the GOP organized the Senate the next January, the Republican right wing deprived Mathias of the coveted chairmanship of the Judiciary Committee, relegating him instead to the chairmanship of the housekeeping Rules and Administration Committee.

Although Mathias said Friday that he did not make the decision to retire until earlier in the week, the punishment meted out nearly five years ago may have made the decision easier.

To a question asked Friday -- if he had ever seriously considered switching to the Democratic Party, as was often rumored -- Mathias said simply: no.

But in 1976, Mathias toyed with the idea of running for president, first as a maverick Republican, and then as an independent. He trooped around the country bemoaning the decline of the two-party system, the Supreme Court's "faltering position as protector against governmental intrusions" and the absence of a national purpose. He provoked enough interest that his name was entered as a candidate for president on the Republican ballot in Massachusetts, but he withdrew the day before the filing deadline.

Later that year, Sen. Eugene McCarthy (D-Wis.) asked Mathias to join him in a third-party campaign, but Mathias declined, concluding that "the obvious disintegration of the old system had not reached the point where there is an equally obvious demand for a new one."

Mathias' decision to dance with the date he came with produced an estrangement that writer Connecticut Walker said made him "too liberal for the Republicans and too Republican for the liberals."

That was not always the case. In 1960, Mathias, then a "well-tailored 38-year-old Frederick lawyer" and freshman state legislator, campaigned successfully for the House of Representatives by denouncing the Democratic incumbent, Rep. John R. Foley, as a "lopsided extremist" who supported legislation favored by "Americans for Democratic Action and other liberal groups." He offered himself as a "down-the-middle" alternative.

But once ensconced on Capitol Hill, Mathias quickly joined the then-influential liberal wing of the Republican Party, whose members looked to Sens. Clifford Case of New Jersey and Jacob Javits of New York for direction. Soon, Mathias was on the receiving end of accusations that he was in the pocket of the American Civil Liberties Union and similarly hued liberals.

Some observers believe Mathias eschewed opportunities to seek national office and a fourth term in the Senate, not because he thought he could not win, but because he lacked the fire in the belly that usually characterizes seekers of high office.

Columnist George F. Will wrote that Mathias was "contemplating a race -- a stroll really -- for the presidency . . . with the languor of the gently bred, like Adlai Stevenson . . . someone who meanders through life. He does not burn with a hard, gemlike flame."

Mathias' aristocratic bearing, tested in recent years by a middle-age paunch, fits the family background: The framed memorabilia on his office wall includes the Union Party ballot of 1864 that lists his great-grandfather, Col. Charles E. Trail, as a candidate for the Maryland Senate on a ticket headed by Abraham Lincoln; and his wife, Ann Hickling Bradford, is the daughter of former Massachusetts governor Robert F. Bradford and a descendant of Pilgrim leader William Bradford.

But the grand lineage did not fit his life style. Charles McCurdy Mathias Jr. -- "I have been called Mac all my life," he explained -- "because my father, grandfather and uncle were also Charlies" -- was born in Frederick on July 24, 1922, and, despite maintaining a residence in Chevy Chase during his congressional days, he has remained a small-town fellow; he prefers to think of himself as a farmer.

In retirement, he is certain to spend many hours on his beloved farm "west of Brunswick." (One of the great deceptions of his political life was revealed when he fell off a horse at the farm in 1977 and it was learned that the property is actually beyond the Frederick County line in West Virginia.)

That same year, Alan L. Dessoff, a Mathias aide, wrote of the retirement of the Mathias' Blue Bomber, a 1966 Buick station wagon whose odometer broke at 195,505 miles. With an often unattentive Mathias at the wheel -- and the scars to prove it -- the car served through two Senate campaigns and numerous trips to the farm, hauling raw fertilizer, live chickens, geese and sheep, and allowed the senator to arrive unceremoniously, and often unnoticed, at political and diplomatic functions at the glitziest locations in the nation's capital.

Current and former Mathias staff members crowded into the ornate Senate Caucus Room on Friday to swap stories about Mathias and applaud his prediction that the present conservatism in national politics will fade. There is, he said, "a tide in politics that will come again, and be in full flood."

That metaphor reminded Carrie Johnson, now a free-lance writer, of a speech Mathias wrote and delivered on the House floor on Aug. 5, 1966 -- after discarding her prepared text -- during early debate on the Fair Housing Act, that she believes eloquently stated Mathias' views. She looked it up later in the day and recited it:

"To those who say 'never' to every necessary change, I would remind them that one of the greatest lessons of history is that when members say 'never' . . . it is an attempt to build a dam or dike across the channels where history flows. Inevitably, all they dam up behind it is a sense of injustice and of inequity . . . . In every case, the pressure behind the dam breaks through . . . ."