The senators had gathered, in unprecedented numbers, in exceptional chagrin yesterday, to make a hateful decision about expelling a brother from their little paradise.

In his usual seat was Sen. Harrison A. Williams Jr., the black-haired, bushy-browed New Jersey Democrat, a member for 23 years, during which he had offended no one and done much good for the poor and the working man. He had an aide on either side, which meant that Sen. Henry M. Jackson (D-Wash.) had to sit in the aisle.

The sense of dislocation was acute. Things were out of joint. The day was raw and gray, and the weather inside the chamber was no better.

Much sympathy attended Williams, who was convicted of accepting bribes and has been sentenced to three years. Fear and loathing attended the Abscam disclosures. Who would be safe from undercover FBI agents hounding members to commit crimes?

The Senate Ethics Committee voted unanimously last August, after viewing copious videotapes provided by the FBI, that Williams' conduct had been "ethically repugnant." But the conduct of Mel Weinberg, the FBI's con man and chief operative, has been infinitely more repugnant, both during and since Abscam.

Much was made of the fact that, on camera, Williams had refused repeatedly to accept a cash bribe. Williams' counsel for the defense, Sen. Daniel K. Inouye (D-Hawaii), a war hero and former prosecutor, added considerable respectability to his cause.

An hour before the Senate-as-jury convened, Inouye sat in his office, with its tropical fish tank and Hawaiian artifacts, and predicted that the issues of Williams' misconduct and that of the FBI could not be separated.

"We wouldn't be tormenting ourselves today, if it hadn't been for the government," he said.

During recent weeks, as Williams had hurried into the chamber at the last minute to cast votes and as his colleagues embarrassedly talked with him about the weather, Inouye has received about 20 senators seeking a way out of the morass.

"The way out," Inouye said impassively, "would be to resign. I am convinced that he is convinced that he is innocent. I am convinced he is innocent."

To Inouye, what Williams did, in boasting that he could get government contracts, was make a fool of himself.

"You don't expel a senator for doing that," Inouye said.

That was the situation until 2:31 p.m., when Howell Heflin, a massive freshman, former Alabama chief judge and the ranking Democrat on the Ethics Committee, stood up by his desk under the eaves of the Democratic side.

Heflin has a man-in-the-moon profile and speaks in a grinding drawl, and nothing in the way he began suggested that he was about to dismantle the Williams defense or to demonstrate once again that it is always a southerner, one with a love of the law, the language and the institutions, who comes forward at such delicate moments in the life of the Senate.

Heflin, it seems, is in the tradition of John C. Stennis (D-Miss.), who gracefully put away Joe McCarthy; and retired North Carolina Democrat Sam Ervin, who instructed Richard Nixon on the Constitution.

Heflin opened with the usual melancholy expressions about the "pain" and "distastefulness" of the task, elaborate compliments to the chairman of the Ethics Committee and the leadership of both parties, protests of bipartisanship and the threadbare claim that "a public office is a public trust."

But 2 1/2 hours later, he had shown that it is possible to separate the crime of the FBI from the crime of Sen. Williams. He had gone through the evidence point by point, condemning Williams out of his own mouth. He based his devastating conclusions not on the findings of the courts, or the character of Mel Weinberg, but on what Williams himself had told the Ethics Committee.

Williams, Heflin said in a dozen different ways, didn't have to do what he did, didn't have to hold seven meetings with the real and fake rogues to whom he bragged about his influence.

"Absent a showing of overborne will, misconduct by the government--though reprehensible--has no bearing on a proceeding in which we are evaluating whether conduct is befitting the dignity and honor of the U.S. Senate," Heflin said.

The members fidgeted, dozed, squirmed. Williams slumped lower in his chair, chewing on the frames of his glasses. His blonde wife, in purple, looked on from the gallery. The eloquent mountain continued to spew facts, charges, refutations in meticulous, masterly detail.

About 5 p.m., he wound up:

"If a member of this body really knows right from wrong . . . then that member would not hesitate to get up and walk--walk away from sleazy characters swearing like sailors, walk away from talk about sheiks and deals and hiding interests and protection and concealment . . . .

"But he didn't. He stayed, he discussed, he agreed, he promised, he pledged: to abuse his office, his public trust, for which now he must be expelled."

Heflin had robbed an expulsion vote of the "pain."