The Maryland Court of Special Appeals has ordered a new trial in a five-year-old lawsuit brought against Congressional Country Club by one of its members, who has accused the prestigious Potomac club of withholding the wages of some of its employes.

The action by the state's highest court overturns a dismissal by Montgomery County Circuit Court Judge John J. Mitchell of a suit pursued by George Koch, a maverick member of the club for 22 years.

Koch said yesterday he was "absolutely delighted" by the action. He seemed to have lost little enthusiasm for a fight that has cost him thousands of dollars since he initiated it in 1977. "I really look forward to the opportunity of trying the case on its merits," he added.

Kim Saal, general manager for Congressional, had little to say. He noted that "the judge several years ago admonished both sides" that he did not want " 'a trial through the media' . . . and we continue to adhere to that," Saal said.

Koch, president of the Grocery Manufacturers of America, a trade association, filed the suit demanding to see the club's books and records to support his claim that some employes had been "shorted" on their paychecks.

Last October, Judge Mitchell dismissed the case, saying he agreed with the contention of an attorney representing the club that Koch had forfeited his right to inspect Congressional's books because of a "vendetta" Koch had against the club, along with a habit of trying his case in the media.

But the Court of Special Appeals disagreed this week, saying that the basic question before the lower court was not "to determine factual disputes or to determine issues of motive and intent," but to decide whether Koch "had an improper purpose," which would have barred his right to inspect the club's records.

No date has been set for a new trial.

James Thompson, the attorney who has represented the club throughout the dispute, said he had hoped the case would have ended at the appeals court, adding that "we have made four offers to give [Koch] access to the records, under certain conditions and the court of appeals acknowledged that."

When the Maryland attorney general's office reviewed Congressional's records, it found that Congressional had withheld a fixed percentage of its employes' paychecks since 1948. But the statute of limitations had expired by then, barring the state from prosecution.

An assistant attorney general concluded later that "there had been no evidence . . . which indicates that the club had been skimming wages and kicking them back . . . or using them for any other nefarious purposes."

In its opinion, the Court of Special Appeals said much of the time spent on this case could have been avoided if Koch had been granted a degree of access to the club's records.