A federal judge yesterday refused to dismiss the federal antitrust suit against American Telephone & Telegraph Co., charging that the two sides circumvented federal law by failing to provide him with a formal chance to review its terms.

U.S. District Court Judge Harold H. Greene, obviously angry about plans of the company and the Justice Department to end a case he has overseen since 1978, rejected papers the parties had filed to end one of the biggest antitrust cases in American history.

The antitrust trial, therefore, remains in recess until Greene has had an opportunity to study the proposed settlement, he emphasized. Another federal judge Monday blocked a proposal by the two sides to transfer the agreement from New Jersey to Greene.

"This case is too important to the judicial process, to the public and to the country" to be closed on such a "haphazard basis," Greene told a packed courtroom.

Greene also suggested that he might continue the trial of the case if not granted authority over the settlement.

Although most legal experts say there is little chance that the procedural problems with the agreement raised by Greene yesterday could block the landmark settlement, his criticism of the process adopted by the two sides and his curt response to an AT&T assertion that the case was closed appeared to embarrass both sides.

After lead AT&T attorney George Saunders told Greene during a discussion that the case had been dismissed, Greene responded sharply. "Oh, no it was not dismissed," he said.

"This case is history," Saunders replied. Saunders later said the settlement "had nothing to do with what was going on in the court."

Yet Greene also praised lawyers for both sides, and said he was "delighted" that they had reached a settlement. "It's better to have the parties have a hand in their destiny," he said.

The suit, brought in 1974, charged AT&T with violating antitrust laws, and called for a breakup of the company. The settlement, announced Friday, calls for the split sought by the government, with AT&T forced to divest itself of 22 local telephone companies with assets worth about $80 billion. In return, the suit was to be dismissed.

At issue is a plan approved by the two sides under which AT&T and the government would agree to dismiss the case before Greene and file the settlement pact before U.S. District Court Judge Vincent P. Biunno in New Jersey, who oversees a 1956 consent decree between AT&T and the Justice Department.

Biunno was asked to transfer authority for the new decree to Greene, at which point Greene would accept public comment on the decree. That plan collapsed when Biunno approved the decree and refused to issue a decision on transferring the matter here.

What particularly irked Greene was the fact that the two sides had effectively avoided the formal procedures applying to antitrust settlements contained in a 1974 act designed to provide federal judges the opportunity to seek public comment and adopt or reject such agreements.

A number of trade associations and other communications concerns had filed papers with Greene challenging the procedures, suggesting that the dismissal should be set aside.

Both the Justice Department and AT&T have said repeatedly that they wanted the requirements of the act adopted in this matter, even though it generally applies only to settlements and not modifications of consent judgments. It is that part of the scenario that prompted Greene's anger.

Noting that he had overseen 25,000 pages of testimony since the trial began, Greene also was critical of the fact that the suit's resolution was placed in a jurisdication which had reponsibility for monitoring a decree dealing with the relationship between AT&T and Western Electric Co., AT&T's equipment subsidiary.

Greene called on the two sides to provide briefs outlining how they intended to bring the settlement under his aegis. "The parties created this particular procedural problem," he said. "It is up to them to straighten out the problem. I cannot and will not permit this case to be closed without the scrutiny required by the act."

In brief remarks before Greene issued his statement, the two sides outlined their reasons for adopting the settlement procedures. Gerald Connell, the lead government attorney, said the department had every intention of providing Greene with the full power accorded a judge under the law.

Saunders explained that modifying the 1956 decree, which barred AT&T from entering new unregulated fields such as the computer business, was the company's prime concern in settling the suit. AT&T had "no viable business" in the future without the change, and would be left with a "withering corporation waiting for its demise," he said.

Further, Saunders said, legislation pending in Congress would have put far too many restrictions on the company's operations.

Reaction to the Greene action was generally approving and polite, even from the parties whose scheme he had rejected.

"We're very pleased that Judge Greene is going to expedite procedural matters that need to be tidied up," an AT&T spokesman said. "We'll do everything possible to get the docket transferred to his court by Monday. We're obviously taking a look to see what needs to be done."

Ronald G. Carr, an assistant to William Baxter, assistant attorney general for antitrust, said the government is also "prepared to comply."

Several communications industry officials praised Greene's action. "We'll be able to find out how the deals were made," noted Frank Budelman, assistant executive director of the North American Telephone Association.

Stephanie Biddle, a representative of the Computer and Communications Industry Association, which like NATA had urged Greene to clarify his court's oversight for the agreement, said she was delighted by the decision.

"We wanted the hearing to take place before the judge who has the most recent trial experience," she said. Noting that a number of issues, such as the fees firms will pay to the new local-service companies for use of their telephone networks, need to be clarified in court, she added: "I think these things need to be addressed in a comprehensive settlement."