Former president Richard M. Nixon secretly paid $144,000 to Pentagon "whistleblower" A. Ernest Fitzgerald after Fitzgerald promised not to force Nixon to go on trial for firing him, according to papers filed in the U.S. Supreme Court yesterday.

Nixon has also agreed to pay Fitzgerald another $34,000 if the former president fails to convince the Supreme Court that he is immune from lawsuits like this, the papers said.

"If Nixon loses, he will have to pay an additional $34,000, but he will never have to face a public trial on Fitzgerald's claim," attorneys said in the documents. They described the extra money as "little more than a $34,000 wager" on how the Supreme Court will rule on the immunity question.

Nixon has steadfastly avoided court appearances since he left office seven years ago. His reported agreement with Fitzgerald is the first known time that Nixon has paid money to avoid going to trial.

Fitzgerald, 55, contended in a lawsuit filed in federal court here that Nixon approved his ouster in January, 1970, in retaliation for his widely publicized disclosure of more than $2 billion in cost over-runs in the C5A air transport program. The Air Force contended that his job was abolished in budget cuts.

After a protracted battle with the Civil Service Commission, Fitzgerald was reinstated in a GS17 Pentagon job in 1973, and won $80,000 in back pay. But he complained that his new work did not compare with his old job.

Last March, the chief judge of the U.S. District Court here ordered him reinstated to an equivalent post.

Fitzgerald declined comment yesterday on the agreement with Nixon which, according to court papers, was reached in the spring of 1980, shortly before the case was to go to trial. Lawyers for both Fitzgerald and Nixon pledged to keep terms of the arrangement secret.

Washington lawyer E. Barrett Prettyman Jr., who represented Fitzgerald until last month, refused comment when asked about the agreement. Nixon's lawyer, R. Stan Mortenson, could not be reached yesterday.

Last June, however, both lawyers signed a statement, filed with the Supreme Court, in which they said Nixon and Fitzgerald had agreed "to fix the amount of payments" that Fitzgerald would be entitled to in the case.

They added that "the amount of payment" depended on the justices' decision and on subsequent action in the U.S. District Court in Washington, where the Fitzgerald case originally was filed in 1974. In that statement the attorneys emphasized "the case has not been settled."

Information about the amount of the agreement was included in papers filed yesterday in the Supreme Court by attorneys for former national security adviser Morton Halperin and his family. American Civil Liberties Union lawyer Mark H. Lynch, who joined in the court filing yesterday, declined to say how attorneys for the Halperins learned of the agreement described in their papers.

The Halperins had sued Nixon and other officials for damages for wiretapping the family's telephone without a warrant for 21 months in 1969 through l971.

Last June, the Supreme Court deadlocked on the critical question in the Halperin case, whether presidents and their advisers could be sued for damages when they violate the law.

The 4-to-4 vote automatically affirmed a lower court decision which cleared the way for the Halperins to pursue their claim. But it has no impact on other cases.

At the same time, however, the court agreed to review the damage suit that Fitzgerald brought against Nixon, in which the same question of presidential immunity is raised.

Yesterday, attorneys for the Halperins charged that Nixon had "bought his way out" of a trial with Fitzgerald but hopes to succeed on the remaining legal question of presidential immunity so he can then apply it to the Halperin case.

The Halperins' lawyer has asked the Supreme Court to set aside its decision to review the Fitzgerald case. CAPTION: Picture 1, Richard M. Nixon ... seeks decision on immunity; Picture 2, A. Ernest Fitzgerald ... could get $34,000 more