A Greenbelt realty firm has filed an antitrust and restraint of trade suit against a major co-op housing complex in the town, charging that it is trying to coerce resident-owners to sell their units through the co-op rather than through realtors.

The firm, Nyman Realty Inc., charged in a suit filed in circuit court that when Greenbelt Homes Inc. raised its administrative fee for members recently it was a "subterfuge . . . to coerce the large majority of its members, who have heretofore preferred to sell their rowhouse units through (Nyman) and other realtors, into selling their homes through (GHI), and to penalize those members who sell their houses through (Nyman) and other realtors."

Greenbelt Homes is the largest co-op in Price George's Country. Its masonry and frame one-and three-bedroom rowhouses were built on a 250-acre tract during the Depression and World War II. There are no vacancies in the units, which sell from $12,000 to $30,000.

On June 1, the board of directors of the 1,608 member cooperative association raised the administrative fee it charges when a unit is sold in the co-op complex to $550. The fee pays for expenses GHI incurs in the sale, such as inspection of the property and investigation of the buyer.

Prior to the raise, the co-op charged a $150 fee, plus $60 for inspection, plus a 5.5 percent commission. On an average unit costing $16,000, the total sales charges came to about $1,000.

Under the new system, the charges would be halved. However - and this is the point of contention - a seller must pay the $550 fee to GHI even if he or she elects to sell a unit through an outside realtor.

Royal D. Breashers, GHI's general manager, said the directors' intention was to benefit the members by keeping total sales prices - and consequently taxes - down by eliminating commissions. No member, he added, has yet objected.

Nyman Realty contends it has lost half of its listings of GHI units since the decision was made last January to change the fees system in June.