The Federal Communications Commission yesterday ordered Communications Satellite Corp. to refund about $38.8 million to customers for what it alleges was four years of overcharges.

The commission in effect forgave some of the alleged overcharges, which it had estimated at about $62 million in April, saying that the rates may have been justified by economic conditions at the time.

An agreement must be worked out with the long-distance companies doing business with Comsat to pass the money through to their customers. Refunds to individuals are unlikely, one FCC official said; more likely, future rates will be cut.

Comsat declined to comment on the ruling, but in the past has argued that the FCC has no legal authority to order refunds. By law, it can appeal the decision to the commission itself or in the federal court system.

The FCC asked that a lump sum be paid after the pass-through issue is settled. The payment would be a major financial setback for Comsat, representing more than half of its $59 million in after-tax earnings in 1986. Sales that year were $467 million.

Comsat represents the United States in a global communications satellite system called Intelsat. Its business consists largely of selling access to the satellites for the long-distance transmission of voice traffic, data and television programming.

The FCC contends that Comsat set its rates at levels that by the company's own projections would yield a greater return than the federally authorized level of 12.48 percent on assets.

In April, FCC officials concluded that Comsat might owe as much as $62 million in refunds for the period Jan. 1, 1983 to July 31, 1986. In response, Comsat noted special economic circumstances affecting interest rates and said that a future markdown in rates might be appropriate.

The revelation that the FCC might bill Comsat for such huge sums was later cited by Contel Corp., the nation's third-largest independent telephone company, in its controversial decision to back out of a $2.5 billion merger that was being negotiated with Comsat.

In its decision, the FCC commissioners said a refund of $38.8 million would correct for overcharges during the period Aug. 1, 1984 to Dec. 31, 1986. They agreed that the company should not be held responsible for overcharges before Aug. 1, 1984.

Commission officials said their decision was affected by the fact that the FCC had long known that Comsat's rates were excessive and had taken no corrective action.

The rates in question were 60 percent below those that Comsat had been charging earlier, FCC officials say. And in the early 1980s, the legal rate of return for Comsat was lower than those being paid on long-term government bonds.

"It would be reasonable to say that the bureau staff at that time thought that it was a reasonable deal to accept the price reductions even though the tariff was targeted at a rate of return above the prescribed amount," said Gerald Brock, head of the commission's common carrier bureau.

Brock described the failure to change the rate of return as a "procedural error." Had Comsat formally requested clearance for a higher return, Brock said, the commission would most likely have accepted it because of economic conditions at the time.