The Justice Department, rebuffing a personal plea from U.S. Attorney Charles F. C. Ruff, has decided no to ask the Supreme Court to review a controversial U.S. appeals court ruling overturning the murder convictions of two anti-Castro Cubans found guilty in the 1976 car bombing assassination of former Chilean Ambassador Orlando Letelier.

The decision means that the U.S. Attorney's office here must now retry the two men, whose convictions were set aside last September when the U.S. Court of Appeals here ruled that major testimony from two jailmates who had talked to the two men should not have been used at their trial.

"The judgement [by the department] is made and we will now go ahead and pursue the case," Ruff said yesterday. The prosecutor, who, supported eforts to challenge the appellate court decision in the Supreme Court, said he was notified yesterday that the Solicitor General's office, which handles all government appeals to the Supreme Court, had concluded it would not ask the high court to review the Letelier case.

"Now we try to put the pieces back together again," Ruff said of the Letelier case, a complex and sensitive prosecution that lasted for more than five weeks in federal court here in early 1979.

Ruff said that his office has not yet had a chance to evaluate the strength of the government's case without the testimony from the two men who had been inmates with the two defendants in a New York City jail.

The government's key witness, Michael Vernon Townley, an American-born agent for the Chilean secret police force once known as DINA and the man who admitted planting the bomb underneath Letelier's car, would be required to testify again for the government under the terms of a plea bargaining agreement be reached with the prosecutors.The status of other witnesses in the case could not immediately be determined.

Justice Department sources described the decision not to appeal as a close call that was debated at length and only reached after detailed reviews at various levels in the department. These sources said that conflicting recommendations had been made at the highest levels of the department.

Ultimately, Solictor General Wade H. McCree and others in his office concluded that it would not appeal the Letelier decision because the facts and the legal issues involved were so similar to a 1980 Supreme Court case that had resulted in the Letelier reversals, these sources said.

The department's decision also took into consideration that fact that the full appeals court had refused to reconsider the initial decision by three judges reversing the convictions, the sources said.

Former assistant U.S. Attorney Eugene M. Proper, one of the government prosecutors in the Letelier case, yesterday angrily denounced the Solicitor General's decision, which he said "does not take into account the importance of the case or the nature of the crime involved."

". . . The Department of Justice has nothing to lose by asking the Supreme Court to hear the case." Propper said."All that can happen is that they will say no."

Such decisions by the Solicitor General's office ca be taken to the Attorney General's office for further review. However, department sources said that it is extremely rare for an attorney general to overrule a decision by the solicitor. These sources said that two such petitions were brought to the attorney general's office in the last four years and in both cases the solicitor's decision was upheld.

Last September's appellate court decision reversing the convictions and ordering new trials, was based on a Supreme Court ruling in June that prohibited the use of evidence solicited from defendants by fellow jail inmates who are acting as government informants. In a 6-to-3 decision, with the majority opinion written by Chief Justice Warren E. Burger, the court said that such activity would violate the defendant's Sixth Amendment rights to have a lawyer present when questioned by a government agent.

In addition to ordering new trials for Guillermo Novo Sampol and Alvin Ross Diaz, both of whom were convicted of murder, the federal appeals court also ordered a new trial for Guillermo's brother, Ignacio Novo Sampol, who faced less serious chargers of perjury and concealing information about the crime from federal prosecutors.

The appeals court said U.S. District Judge Barrington Parker's refusal to give Ignacio Sampol a separate trial prejudiced his defense case.

In that same decision, the appeals court, ordered Parker to release Ignacio Sampol on bond, which Parker later set at $25,000. The other two defendants also have requested to be released on bond, but Parker has not ruled on the issue.

Much of the prosecution case rested on testimony from Townley, now serving a 40-month-to-10-year prison term after being convicted of murder in the case, and other evidence collected during a 22-month investigation. Five Cubans, allgedly recruited by Townley, and three Chileans were indicted in August 1978 in connection with the murders of Letelier and an assistant, Ronni Karpen Moffitt. The Chilean government has refused to extradite the three Chileans, all former DINA officials, to face charges in the U.S. that they either ordered or helped carry out the killing of Letelier, a prominent official in the overthrown government of Marxist Salvador Allende and who in exile was a vocal critic of current Chilean dictator Augusto Pinochet.

Both Letelier and Moffitt were killed when a car Letelier was driving exploded at midmorning on Embassy Row in what has been considered the most notorious act of international terrorism ever committed here.