A federal appeals court yesterday upheld a lower-court ruling that American Telephone & Telegraph Co. violated antitrust laws in dealings with MCI Communications Corp., but rejected a jury's award of a record $1.8 billion in damages to MCI.

The 7th U.S. Circuit Court of Appeals in Chicago ordered a new trial to determine the amount of the damage penalty to be paid by AT&T to MCI in the lawsuit, which accused AT&T of trying to stifle competition.

The three-judge appellate court threw out two of the 10 antitrust violations found by a jury in June, 1980. The appeals court said the case now must be retried to determine damages on the eight antitrust violations that it upheld.

The long-awaited action leaves both companies with the options of asking the appeals court to reconsider its decision, of asking the Supreme Court to consider the case or of returning to District Court for retrial of the damages portion of the jury verdict.

In the original lawsuit, MCI, which is headquartered in Washington, claimed that AT&T violated federal antitrust laws in denying MCI access to various local and long-distance services needed to develop its business and in setting discriminatory prices on its services.

MCI had asked the court for a $900 million antitrust award. If granted, this would have resulted in a $2.7 billion penalty, as federal antitrust laws provide for penalties up to triple the amount of actual damages. The jury ruled against AT&T on 10 of the 15 allegations it had considered and granted an award amounting to two-thirds the damages MCI had sought.

Both AT&T and MCI claimed yesterday's decision as a victory. An AT&T spokesman said the company is "pleased that the appeals court has set aside the judgment and all of the damages have been reversed."

"The heart of the case--the pricing issue--has been thrown out," the company said. "What remains are minor matters."

MCI Chairman William McGowan said his company read the decision differently, noting that, unless the appeals court decision is overturned, "we'll never have to prove liability again.

"They were found guilty of violating the antitrust laws," he said in an interview. "That will be very helpful to a jury" in awarding damages.

"They should pay and should pay dearly," McGowan added.

Some Wall Street analysts had expected the damages portion of the case to be overturned, and noted that MCI's corporate plans had not counted on getting the money.

Steven Chrust, an analyst with Sanford C. Bernstein & Co., said it "appears that it has to be described as at least a moral victory for MCI." Chrust noted that with MCI now recording revenues of $1 billion a year, the company can "say to a jury that we would have been here sooner."

Charges that were upheld by the court include the jury's finding that AT&T did not provide proper equipment or procedures to permit MCI to connect its long-distance microwave network to AT&T's local phone system.

The court also upheld allegations that AT&T negotiated in bad faith with MCI and improperly filed state rate schedules designed to hinder MCI. Further, the court upheld the charge that AT&T illegally failed to provide specialized long-distance links that MCI had requested.

The MCI suit provided a principal building block for the antitrust suit brought by the Justice Department against AT&T, which resulted last year in a settlement calling for the breakup of AT&T in 1984.