A federal appeals court here yesterday set aside the Federal Communications Commission's rules restricting the type of shows that can be broadcast over pay cable television.

In overturning the 2-year-old regulations, the three-judge U.S. Court of Appeals sharply criticized the FCC's reasoning behind the cable TV regulations, the agency's justification for policing the content of cable television, and the manner in which the specific cable TV rules were adopted.

The court found that the FCC had improperly heard private comments from parties involved in the pay cable television issue. It told the commission that its proceedings were so tainted with these secret comments that it should start anew in considering any pay television regulations.

It is unclear what immediate effect the ruling will have on the burgeoning cable television industry. The use of coaxial cables, instead of standard lows companies to provide to home television viewers, for a small monthly fee, a wide range of special events that are not available on commercial television channels.

Approximately 1 million persons already subscribe to more than 350 cable television systems across the United States, according to FCC and cable television industry representatives.

In the Washington metropolitan area, about 5,400 persons subscribe to Reston's struggling cable television company to view regular programming from Washington and Baltimore's television channels as well as local Reston programming and first run movies.

Cable television industry representatives praised yesterday's ruling, which appears to re-open the whole spectrum of the types of programs that can appear on their outlets.

"We're elated with the decision. We think it's a true reflection with what we've said all along - the FCC should not be constraining the marketplace, said National Cable Television Association President Robert Schmidt.

An FCC spokesman declined comment on the lengthy opinion late yesterday, Saying the agency's attorneys were studying it.

The stated purpose of the FCC rules was to prevent popular special events from being "siphoned" away from the regular television markets in competitive bidding situations between regular networks and pay cable television companies.

In its unsigned opinion the U.S. Court of Appeals said there was no evidence as to such "siphoning" and that therefore the FCC had no evidence to support its alleged need to regulate the pay cable television industry.

Even if the agency had the specific authority to regulate pay cable television and the evidence to do so, it acted improperly in passing the rules to prevent "unfair competition" between conventional and cable television, the court added. The Justice Department also had challenged the FCC's rules.

The court said that "at a minimum the commission, in developing its cable television regulations, was required to demonstrate that the objectives to be achieved by regulating cable television are also the objectives for which the commission could legitimately regulate the broadcast media."

The court found further that the FCC "had in no way justified its position that cable television must be a supplement to, rather than an equal of, broadcast television . . . We cannot fathom how the commission reached the conclusion that the balance here should be struck in favor of regulation" of the cable television industry.

The three judges - U.S. Circuit Court Judges J. Skelly Wright and George MacKinnon and U.S. District Court Judge Stanley Wiegel - also found that he rules violated the First Amendment by "serving no purpose" in prohibiting the showing of certain events.

Much of the court's opinion focused on the alleged improper contacts between parties in the case the FCC commissioners and staff. Others not directly involved in the case - including members of Congress, members of the trade press and representatives of various performing arts groups - also made questionable contacts to discuss alone and in confidence rather than in public the merits of the rules being reviewed by the FCC, the opinion said.

"Even the possibility that there is here one administrative record for the public and this court and another for the commission and those 'in the know' is intolerable," the judges said. They said commissioners and staff members must in the future record in some fashion their meetings with industry representatives when they discuss pending rules.