The name of Richard Nixon once again dominated a Supreme Court debate yesterday and once again it produced a contentious argument, with several justices expressing irritation over a deal made by the former president to avoid standing trial in a civil suit.

Some justices suggested that the deal Nixon made with fired Pentagon "whistle-blower" Ernest Fitzgerald may have mooted the issue before the court--presidential immunity from civil suits. If so, it would be the second year in a row this major controversy has remained unresolved.

Fitzgerald sued Nixon and two Nixon aides, Bryce Harlow and Alexander Butterfield, over a 1973 Nixon administration decision that cost him his job. Fitzgerald charges that the decision was in retaliation for testimony he gave before a Senate committee exposing massive cost overruns on the C5A military aircraft.

In May, 1980, Nixon agreed to pay Fitzgerald $142,000 on the condition that however the Supreme Court ruled, Fitzgerald would not take the former president through the public spectacle of a trial. In addition, Nixon and Fitzgerald agreed that if the court ruled in Fitzgerald's favor, Nixon would pay him $28,000 more.

Attorneys on all sides conceded that there will be no trial for Nixon, no matter how the justices rule.

"What are we addressing here?" said a visibly irritated Justice William J. Brennan Jr. at oral arguments yesterday. "What are we here for?"

"There's still that payment of $28,000" at stake, responded Nixon's lawyer, Herbert J. Miller Jr.

"That sounds to me like a feigned issue," Brennan snapped, "a feigned issue to get us to rule, to render an advisory opinion."

Justice Sandra Day O'Connor put it a different way: "It's almost a wager on how this court will rule, isn't it?"

"I wouldn't so characterize it," said Miller.

Elliot L. Richardson, who was fired by Nixon as attorney general in the "Saturday Night Massacre," was ironically on Nixon's side yesterday, representing Harlow and Butterfield. He pointed out that his clients are not part of any deal and that the court's opinion will determine whether or not they stand trial.

Richardson argued that if the president is immune from Fitzgerald-type suits, his aides should be immune as well. But even if the court does not rule on presidential immunity, he said, aides perform their own "special functions" and should be entitled independently to immunity.

At stake in the Fitzgerald case is also a money damages suit brought against Nixon and other Nixon assistants, including Henry Kissinger, by former national security adviser Morton Halperin. Halperin won his case in the U.S. Court of Appeals for the District of Columbia after getting an unprecedented ruling that presidents and presidential aides are not immune from lawsuits.

The Supreme Court affirmed the Halperin victory last term. But because it did so on a tie vote (Justice William H. Rehnquist, a Nixon-era Justice Department official, did not participate), the ruling left the issue unresolved.

In the Halperin case and in the Fitzgerald case, Nixon lawyers and the Justice Department argue that absolute immunity is essential for the smooth operation of the presidency. Without it, they say, presidents and their aides could be harassed by legal proceedings both during and after their terms.

Fitzgerald's lawyer, John E. Nolan Jr., and civil liberties activists contend that the suits are an effective deterrent to unconstitutional and illegal White House behavior.