Bill Scott sits in the same high-backed leather chair that was his on the floor of the Senate. He works behind the same massive mahogany desk he used in the Dirksen Building, flanked by the flags of Virginia and the United States. But his favorite object in the study of his Fairfax County home is the gleaming white marble fireplace, an exact replica of the one in the Senate conference room.

"There's a cliche: once a senator, always a senator," said 66-year-old William Lloyd Scott, the Virginia Republican who retired in 1978 after one term in the Senate and three terms in the House representing Northern Virginia's 8th Congressional District.

Ten years after he ousted a moderate Democratic senator, Scott may still be best remembered as the lawmaker who struggled to overcome the label, pinned on him by New Times magazine and a Ralph Nader research group, of being the dumbest congressman and one of the least effective.

"The press hasn't been very accurate or very kind," said Scott. "Reporters tend to be more liberal than I am."

Scott now spends his days with his wife Inez in the $271,000 English Tudor-style home they built last year on six secluded acres hear Fairfax Station and named "Leisure Hill." They see a few friends, tend to their 150 azalea bushes and spend time with their grandchildren.

Most evenings they sit in reclining chairs in their panelled basement recreation room, its walls dotted with photographs of them, former President Nixon and former vice-president Spiro T. Agnew, and watch television.

"We're very happy being retired," Scott said.

Scott, who quickly dropped from public view after his retirement, has not dropped out of politics entirely. Last spring he rejoined the Fairfax County Republican Committee after a long absence, for the purpose, he said a bit wistfully, of "getting back with some of our old friends."

Although his tenure was widely viewed by many in his own party as less than successful, Scott accomplished something no Virginia Republican had done since Reconstruction: He got elected to the Senate, knocking off incumbent William B. Spong, then regarded as unbeatable.

"Bill Scott's victory in 1972 helped build momentum for the party," said Virginia's former Republican Gov. Linwood Holton, now a Washington lawyer. "He showed Republicans that anybody could win."

Scott's victory, which he regards as his finest achievement, was proof of the viability of a state GOP long eclipsed by the dominant Democratic Byrd machine.Yet, for a man whose stunning upset victory seemed to presage so much, Scott is scarcely regarded as one of the party's elder statesmen. He wields surprisingly little influence in Virginia, a state that worships tradition and tends to venerate its political elders.

Three weeks ago, he went largely unnoticed during the two-day GOP state convention in Richmond that nominated the party's current Senate candidate, Rep. Paul S. Trible Jr. Two years ago, despite his early support for President Reagan, Scott unceremoniously was bumped as a delegate to the Republican National Convention.

"He's certainly no kingmaker like Mills Godwin or John Dalton," said one GOP strategist, referring to the two former Republican governors. "He still shows up at events and people are polite, but nobody really talks to him. The new people in the party don't know who he is, and many who do would just as soon forget."

While Scott's impeccably conservative voting record made him popular with many Virginians, his irrascible manner, verbal gaffes, overseas junketing and proclivity for offending potential supporters received wide publicity, to the embarrassment of some party leaders.

After a visit to the Panama Canal, one of 38 countries he visited at taxpayers' expense during his single Senate term, Scott said he had seen "ships going both ways." He once chided a group of reporters for failing to stand when he was introduced, saying it showed disrespect, and adding: "There are those who say it helps to straighten out your underwear."

On another occasion, Scott walked out of a meeting with black residents of Arlington, saying, "The great bulk of the colored people are going to vote for the Democratic presidential nominee anyway."

"I have no desire to reenter the Washington scene," said Scott, who left Congress after 38 years as a federal employee and now receives an annual pension of $61,000.

Because he worked for the federal government for so long, beginning as a 19-year-old messenger at the Government Printing Office, Scott's pension is about $6,000 more than his Senate salary. "I was working for nothing," he said of his years in the Senate.

Following his retirement, Scott spent a year practicing law with his two sons in Springfield. The firm specializes in highway condemnation cases. "After being in the Senate, it was a little boring to check descriptions on legal documents," recalled Scott, who worked his way through college and night law school at George Washington University and spent 18 years as a condemnation attorney with the Justice Deparment.

Scott does not miss the erratic hours required of a Senator. "Some of the senators used to kid me a little bit because I said on the floor, 'Okay, it's eight o'clock, let's go home,'" he related. "That was one of the worst things about the job, never knowing when you'd be able to go home."

There were other frustrations. When Scott left Capitol Hill, Democrats controlled both houses of Congress and the White House. His conservative Republican colleagues, then so out of favor, are now powerful committee chairmen.

"When you're in a minority, it doesn't encourage a person to attempt to get his ideas enacted," said Scott, reclining in a leather armchair before the fireplace in his study. "Most of the time you're working in opposition to what the other party's trying to do. There is a little bit of envy in not having served at a time when the Republicans are in the majority."

Scott does not like reflecting on his 12 years in Congress. He looked baffled when asked to name the piece of legislation of which he is proudest. "I don't know of any," he said, frowning slightly. Pressed, he cited his role as floor leader of the opposition to the Equal Rights Amendment.

"We tried to serve our constituents and maybe we were a little overzealous in that respect," recalled Scott, looking trim in a white polo shirt and blue-and-white striped seersucker pants, his gray hair swept straight back. "Possibly I should have left more of that to the staff, but people want to talk to the officeholder. After all, your duty is to serve the public."

He looked equally puzzled when asked how retirement has changed him. "Well," he said as he wrapped his napkin around a glass of iced tea embossed with the seal of the Congressional Country Club, "I've lost 15 pounds." He placed his glass near an ashtray bearing the signature of Nixon, whom he will not discuss.

"The Watergate thing was 10 years ago," said Scott abruptly. "It's a waste of time to relive the past." He also waves away discussion of his treatment by the press, although the bitterness occasionally surfaces. "I have no desire for publicity," he said in the slow drawl of his native West Virginia.

What he does relish talking about is his 1972 victory over Spong, his one-term predecessor, now dean of the law school at the College of William and Mary. "That was the year Jesse Helms was elected, but he didn't beat an incumbent Democrat," Scott said proudly. "I was the only one to do that in 1972."

Scott won, political strategists say, by wrapping himself in the mantle of Nixon, who then was enormously popular in the state. Scott rode to victory on the crest of Nixon's popularity at the polls, a strong antibusing campaign and a $200,000 television blitz, then unprecedented in Virginia.

"Money didn't win the election, friends did. We had twice as many people working for us as Spong had," said Scott.

"Nobody dared take on Billy Spong in 1972, so when Bill Scott said he wanted to run we said fine," said one high-ranking Republican who requested anonymity. "We were sure he would lose and that way we could get rid of him."

Although Scott proved to be an embarrassment to some, other GOP leaders fondly recall his conservative voting record.

"Bill never, ever faltered," said Bill Stanhagen, a Prince William County lawyer and Republican National Committeeman. "He was a true-blue Republican down the line. But there is something about Mills [Godwin], John Dalton and John Warner, who generate a respect laced with warmth and loyalty that Bill never pulled off."

Scott is remembered as much for what he didn't do as for what he did. "He didn't do any damage to the party," said Dave Foreman, former chairman of the Fairfax County Republican Committee. "He always voted right. People respect him."

Scott still goes periodically to the Hill, stopping at the members' gym and lunching in the Senate dining room -- "the private Senate dining room, for members, not the public Senate dining room," he emphasized. "There's no particular individual I see. You don't like to impose, but you're always made welcome."

Several times a year, the Scotts attend conferences held by an association of former members of Congress. "We spend most of our time at home," Scott said. "We do a lot of manual labor and we rest a lot . . . We don't need to take any sleeping pills or tranquilizers anymore and we don't have the tensions we had in public life. Nobody bothers us here."