Not since 1798 and the days of John Adams, when one Rep. Matthew Lyon of Vermont was chastised for committing a "gross indecency" after he spat on Roger Griswold of Connecticut, has Capitol Hill buzzed so loudly with the sweet sounds of partisan battle. Call it the Downey-Dornan Duel, and yesterday there was talk of it all over Washington.

"Congressman Dornan owes me and the House of Representatives an apology," Rep. Thomas Downey (D-N.Y.), 36, asserted yesterday.

"Apologize for what? For calling him a wimp?" responded Rep. Robert Dornan (R-Calif.), 51. "I am willing to concede that perhaps he just walks, talks and acts like a little arrogant wimp. But maybe it's disinformation. Maybe he really wears a black leather jacket by night that I don't know about."

It started innocently enough last Saturday. Dornan, a conservative, called Downey, a liberal, a "draft-dodging wimp" in a speech before the Conservative Political Action Conference, because of Downey's repeated criticism of antigovernment "contra" forces in Nicaragua. Downey received a 1-Y medical deferment from the draft because of a perforated eardrum. He had actively protested against the Vietnam war.

According to several firsthand accounts, Downey confronted Dornan on the House floor Monday afternoon, attracting an audience of a dozen members and rekindling a two-year feud.

Dornan claims Downey grabbed him by the shoulder and turned him around, wanting to know if he had indeed called him a wimp.

"I did and you are," Dornan said he told Downey.

Downey wasn't talking about the fight yesterday, only releasing a statement asking for the apology. Earlier, he was quoted as saying that when he tried to walk away from Dornan after the heated exchange, Dornan grabbed him by the tie and collar threatened him with "bodily harm."

Dornan says he just wanted to straighten the knot in Downey's tie. "I like the members to look elegant on the floor, you know," Dornan said yesterday.

"Dornan grabbed Downey roughly by the collar, and I mean aggressively," said a Democratic witness, Rep. Mike Lowry of Washington state. "None of this straighten-the-tie baloney. And he told Downey, 'Don't let me catch you off the floor, where you are protected by the sergeant at arms.' I really think Downey restrained himself."

Dornan, according to witnesses, then told Downey to "get out of my face." Dornan confirms he said that.

"I found it humorous that Downey had to run up to the Speaker when it was over to tell all," said Rep. Chris Smith of New Jersey, a Republican witness. "It was like a little classroom act . . . Very childish . . . I think he's made much to-do about nothing."

Downey said through a spokesman that he would not speak to reporters "until Dornan apologizes." The spokesman said Downey was considering filing a complaint with the House Office of the Sergeant at Arms.

"What's he going to complain about? That I wrinkled his tie and he wants me to pay the dry-cleaning bill?" said Dornan.

The Dornan-Downey battle apparently goes back two years, when Dornan, not then in Congress, was up for an administration appointment at the Arms Control and Disarmament Agency and Downey spoke out against it on the House floor. The administration withdrew the offer. Monday, during the fight, Dornan accused Downey of costing him the job.

"It was still in Bob's craw," said Brian Bennett, Dornan's administrative assistant.

House Speaker Thomas P. O'Neill Jr. (D-Mass.) said yesterday he would not tolerate such behavior.

"You can settle it on the street, but don't settle it on the House floor," O'Neill said he told Dornan Monday. The House speaker also told reporters yesterday there would be "discipline" if there were "anything like that" again.