Martin-Marietta Corp. has decided to ask the Pentagon to pay the legal costs of its unsuccessful libel suit against The Washington Star for publishing an article it said was false.

The aircraft maker will contend that the money was a normal cost of doing business with the Pentagon as a defense contractor.

A senior Pentagon official said in a telephone interview it was "doubtful" that the government would agree to pay the costs.

Lawyers for The Star said they were astounded that the government would even consider underwriting a defense contractor's legal costs in a controversial libel action initiated by the contractor. The Star's lawyers estimated that Martin-Marietta paid $175,000 to $200,000 in legal fees for its case, which was dismissed by a federal judge before it came to trial.

The Pentagon does not pay legal fees in cases that are deemed part of a comparty's normal "overhead" in fulfilling a Defense Department contract.

Sen. William Proxmire (D-Wis.), self-styled spending watchdog, has twice written Pentagon officials to ask if the department would allow the fees for this libel suit to be considered as "overhead."

The latest reply, from Dale W. Church, deputy under secretary of defense, convinced Proxmire and his aides that the Pentagon was inclined to finance the Martin-Marietta libel suit.

The letter noted that the cost of a libel suit is not specifically precluded from calculations of acceptable "overhead," and said that if the fees satisfy other criteria "they will be considered allowable."

But in a telephone interview, Chruch said he thought the letter pointed toward Pentagon rejection and said it was "doubtful" that any claim would be approved.

This suggests that Pentagon doubts that the costs of the libel suit were "incurred in the ordinary conduct of the contractor's business" or were "reasonable in nature and amount," the general criteria listed in Church's letter to Proxmire.

The libel suit involved an article by Peter Grunenstein, then with Capitol Hill News Service, that was published in The Star and other newspapers in 1975.

Gruenstein reported that Pentagon officials had attened a party for which prostitutes had been hired to entertain the guests at a Wye Island, Md., hunting lodge leased by Martin-Marietta, and that one of the prostitutes had received $3,000 from a Martin-Marietta representative.

At noon the day the article ran, the chairman of Martin-Marietta telephoned the managing editor of the Star saying the account was completely wrong, asking him to kill it in later editions, and saying company lawyers were looking into possible libel action.

The Star did not kill the article or print a retraction, and the firm filed suit. The Star's lawyers argued that Martin-Marietta should be considered a "public figure" in the terms of a landmark Supreme Court libel case, which meant it could only be libeled if the author published his story with "actual malice," knowing that it was false or with "reckless disregard of whether it was false or not."

U.S. District Court Judge Thomas A. Flannery agreed, and dismissed the case. However, much legal maneuvering and deposition-taking preceeded his decision, running up substantial legal bills for both parties.