The Supreme Court refused yesterday to hear an appeal by six real estate firms that were convicted of conspiring to fix the rate of commissions on homes sold in Montgomery County, a plot that brought the firms an additional $700,000 in fees over a two-year period.

In letting stand a 1977 criminal conviction by a U.S. District Court in Baltimore, the Supreme Court cleared the way for the Maryland Real Estate Commission to suspend or revoke the licenses of the firms, some of which are among the largest in the Washingtonarea.

A spokesman for the commission said it is "likely" that a hearing will be scheduled soon. The high court's refusal to grant a hearing to the companies ends a legal process that already has resulted in fines of $200,000 for the companies and an agreement that will allow the homeowners who paid the higher commissions to get a discount rate the next time they sell a home.

The trial court found, and an appeals court later agreed, that the six companies and three of their presidents were guilty of violating anti-trust laws when they met at the Congressional Country Club early in 1974 and decided to raise their commissions from 6 percent to 7 percent, an action that boosted their income by more than $700,000 on sales of $350 million between 1974 and 1976.

Although technically the rate a real estate agent can charge is negotiable in Maryland, the common charge for many years has been 6 percent. Before the 1974 meeting of the six firms, fewer than 3 percent of the more than 300 brokers in Montgomery County charged 7 percent commissions. Within a year of that meeting nearly 40 percent of the county's brokers had gone to 7 percent commissions.

The six companies involved were Jack Foley Inc., Colquitt-Carruthers Inc., Bogley Inc., Robert L. Gruen Inc., Schick & Pepe, and Shannon & Luchs Co.

After the convictions, U.S. District Court Judge C. Stanley Blair imposed fines, but did not order prison sentences, although federal prosecutors had argued for stiffer sentences, including jail for two of the brokers.

Yesterday's decision was foreshadowed two weeks ago when, in an unrelated civil action, the court ruled that New Orleans real estate brokers were subject to the laws of interstate commerce. The Montgomery County brokers also had contended that their businesses were not covered by those federal regulations.

John Lewin, attorney for John P. Foley, said the court's refusal to hear an appeal "will make realtors aware that there's an antitrust law, but I doubt we will see a tremendous disparity (in commissions) between leading firms.'"

Lewin said he thought there were "serious legal issues raised" by the appeal. "We argued that there had to be a showing by the government that the purpose was indeed to fix prices."

The court does not issue an opinion when it refuses to grant an appeal.

In the New Orleans case, however, Chief Justice Warren E. Burger wrote that although buying and selling a home is a local transaction, the business is intertwined in interstate commerce because mortgage financing, title insurance and loan guarantees often involve out-of-state companies.

In addition to fining the six companies Judge Blair also fined Jack P. Foley Jr., president of the firm bearing his name, $10,000, and placed him on probation for three years; fined John T. Carruthers Jr. of Colquitt-Carruthers $25,000 and placed him on probation for three years, and fined Robert W. Lebling, president of Bogley, $5,000 and ordered one-year probation.

The fines levied on the companies were: Foley, $15,000; Colquitt-Carruthers, $50,000 Bogley, $15,000; Gruen, $15,000; Schick & Pepe; $25,000 and Shannon & Luchs, $40,000.

After the convictions, a number of civil actions were initiated, including one by the State of Maryland in behalf of the sellers who had been charged the higher commissions.

As a result of that class action suit, about 1,800 certificates will be issued to those homeowners entitling them to list their home at a 5 percent brokerage fee the next time they sell.

The certificates, which amount to a saving of $1,000 on the sale of a $100,000 home, are negotiable and can be used two times.