For Sol Rosen the congressional sex scandal he helped bring to the nation's front pages three years ago is finally over. Rosen is the lawyer who in 1976 encouraged his client (and sometime girlfriend) Colleen Gardner to go public with charges that she was paid $25,800 by Rep. John Young (D-Texas) primarily for sexual entertainment.

Gardner's charge-coming on the heels of a similar accusation against another congressman by an old acquaintance of hers named Elizabeth Ray-gave impetus to a summer of scandalmongering. Young, meeting the press behind dark glasses, denied the charge while acknowledging that he had held some clandestine meetings in area motels. But he said he was huddling with military officials who passed confidential information to him.

Apparently his constituents didn't believe him; Young lost a reelection bid in 1978. Two years ago his wife committed suicide. And last month a federal judge in Washington dismissed his $6 million libel suit against Rosen and The New York Times.

"It sort of vindicated everybody," says Rosen of the dismissal of Young's suit. Now he's considering dropping his suit against Young, a suit that charges the congressman with libel for statements he made against Rosen shortly after Gardner's allegations hit The New York Times.

"We deposed Young for three days," says Rosen. "He admitted his relationship. He admitted he'd never gone to motels to meet military people. He denied he raised her salary for sex. He gave rambling answers. He'd arrive with abriefcase that held a flask of whiskey and a bottle of Listerine."

Young, who still lives in McLean, refused comment on all aspects of the cases referring reporters to his lawyer, Thomas Rothwell, who also dosen't comment to the press.

Rosen, 43, is a rotund man who talks with machine-gun bursts of words. For 16 years he's practiced law solo, mostly representing indigent defendants. In 1971, for example, he earned a whopping $70,000 in federal fees fore that kind of work, representing 300 defendants in 230 long workinng days. The Sol Rosen doll, joked colleagues, worked this way: wind it up and its client pleads guilty. Because of Rosen, legislation was fashioned that forbids a single lawyer from earning more than $17,000 a year from court-appointed cases.

That's fine with Rosen, who says he's tired and disillusioned wirh on-the-run justice.

"No one really cares," he says. "You suddenly realize you're just part of the system like an automaton. No one cares, not even the client. So I win a casr on search and seizure. Big deal. In two months the guy is arrested again."

Rosen-who keeps men's magazines on his coffee table, calls men "buddy" and women "honey," and dresses in '50s-style clothes that somestimes clash-says now he's concentrating on areas of the law that include employment and housing discrimination. He doesn't keep in touch with Gardner, who may-Rosen doesn't want to be too specific about this-have changed her name to shed the notoriety the Young scandal brought her.

Rosen instigated his suit against the congressman because he felt the Justice Department was dragging its feet in devloping a case against him. The lawyer rails against federal presecutors who decided they had insufficient evidence to prosecute Young without even interviewing the congressman. The glare of publicity his suit helped keep on Young during his reelection during effort, Young's admission during depositions, and the dismissal of Young's suit against him are meat to Rosen satisfaction enough. Though he adds that the whole scrap "also taught me libel law." CAPTION: Picture, no caption, By Craig Herndon