The Maryland Court of Special Appeals yesterday overturned a ruling by a Montgomery County judge who had ordered the press and public excluded from a pretrial hearing for Timothy Joseph Buzbee, who is charged with several rapes in the Aspen Hill area.

A three-judge panel of the appellate court ruled that the order by Circuit Court Judge Rosalyn B. Bell violated the First Amendment right to freedom of speech and freedom of the press. As a result, the hearing on two motions to suppress evidence, scheduled for Monday, will be open to the public.

Bell's ruling last month--which attorneys said marked the first closing of a pretrial hearing in county history--set off a confrontation between competing constitutional principles: the public's right to know and the defendant's right to a fair trial.

Buzbee's lawyers sought the closed hearing, arguing that the case had generated substantial publicity and that further publicity could make it impossible for Buzbee to get a fair trial from an impartial jury in the county.

At the pretrial hearing, defense lawyers were expected to argue that certain statements and other evidence against Buzbee should be thrown out.

County prosecutors, the Montgomery Journal, the Montgomery Sentinel and The Washington Post opposed Judge Bell's order, asserting that the public has a constitutional right to attend the hearing and that the defense had failed to show that publicity would prevent a fair trial.

Buzbee's attorneys also sought and won from Bell a "gag order" that prohibited attorneys, the police, Buzbee's family and prospective witnesses from talking publicly about the case. The appellate panel overturned that ruling yesterday.

Bell also ordered that certain documents already filed in the case be sealed from public view and that some future documents be sealed until defense attorneys had a chance to review them. Normally, documents in court files are open to the public.

As a result of the appellate panel's order, all documents filed in the future will initially be open to the public. The order, however, did not refer to the documents already sealed.

Defense attorney Reginald W. Bours III maintained that the panel, in failing to mention those documents, was upholding Bell's order to seal them. Carol Weisman, an attorney for The Post, said that the panel was silent concerning that order and the newspapers may seek clarification of that portion of the ruling.

Judge Bell said yesterday she would have no comment on the appellate rulings.

Bours said he will probably seek a brief postponement of the hearing scheduled for Monday until he can determine a further course of action. Bours has the option of appealing yesterday's ruling to the Maryland Court of Appeals, the state's highest court.

Buzbee, 26, was arrested last November and later charged in connection with five of the 16 sexual assaults that had terrorized the Aspen Hill community for 18 months.

Gregory M. Schmidt, who argued the appeal yesterday for the Journal Newspapers, told the court that the Buzbee prosecution was "of great concern to the citizens of Montgomery County and Aspen Hill" and that the defense had mounted "an initiative . . . to shut off the flow of information to the public" from the moment of Buzbee's arrest.

The appellate panel's ruling came in a one-page order, which said that an opinion outlining the court's reasoning would follow. But Judge Alan M. Wilner appeared to indicate some of the court's thinking as he questioned Bours during yesterday's arguments.

"Is extensive publicity enough to close a hearing?" he asked Bours. "Was there not extensive publicity in the Hinckley case and the Harris case?" he added, referring to the prosecutions of John Hinckley Jr., accused of trying to assassinate President Reagan, and Jean Harris, convicted of killing Scarsdale diet doctor Herman Tarnower. Pretrial hearings in both those cases were open to the public.