A move to censure rather than expel Sen. Harrison A. Williams Jr. (D-N.J.) yesterday appeared to be failing for lack of support.

Senate Majority Leader Howard H. Baker Jr. (R-Tenn.) said he had polled the 53 Republican members of the Senate and found almost no support for a censure resolution introduced by Sen. Alan Cranston (D-Calif.), the minority whip.

"The odds don't look good" for Williams, his chief defender, Sen. Daniel K. Inouye (D-Hawaii), said as he walked into the Senate for the fourth day of debate on expulsion.

Baker declined to reveal the precise tally of his head count, but an aide said Republicans were "not unanimous, but near unanimous" in opposing censure.

Censure requires a simple majority of those present and voting, 51 if all are present. Expulsion requires two-thirds' approval of those present and voting.

Without the votes of at least four Republicans the censure move doesn't have even a mathematical chance of passage, even if every Democrat votes for it. Several Democrats indicated yesterday that that is extremely unlikely.

"Sen. Williams has not had the good grace and good judgment to withdraw from this body. We should not perpetuate our own disgrace by asking him to stay" by voting for censure, said Sen. Thomas F. Eagleton (D-Mo.).

Williams, convicted of bribery and conspiracy in last year's Abscam investigation, is eligible to vote on his fate in the Senate.

"My feeling is they have not bought his arguments in this," said Sen. Howell Heflin (D-Ala.), vice chairman of the Ethics Committee, which voted unanimously last summer to expel Williams.

Sen. Malcolm Wallop (R-Wyo.), the panel chairman, said "something truly strange" would have to be introduced to endanger the two-thirds' expulsion majority.

"I think the Democratic party would want to put considerable pressure on him to resign," Wallop added in an interview.

Sen. Minority Leader Robert C. Byrd (D-W.Va.) told Williams weeks ago that he should resign for the good of the party. Williams refused, and he has repeatedly brushed aside such suggestions ever since.

At a news conference after the debate, Williams indicated that he thought the debate is taking partisan tones.

"The Republican position today is solidly in favor of expulsion," he said. "The debate is not over. The matters are not all presented . . . . I think it is fundamentally wrong for judgment to be made before this case is concluded. I hope there is a turnaround."

Comments yesterday came as Cranston formally introduced a strongly worded censure resolution, which is expected to be voted on today or Thursday.

"To censure a senator is not just to slap his wrists" or "set a stamp of approval on Sen. Williams' behavior," Cranston told a two-thirds-full Senate. "To censure a senator is to take harsh action against a senator. Any one of us would feel disgraced if he were censured."

Cranston acknowledged his original reluctance to become involved in the case because he wondered, among other things, "how it would look to the folks back home."

He based his argument for censure rather than expulsion on two assertions: that it would be premature to expel Williams before his appeals are complete, and that there was "the grossest abuse of power and misconduct" by the executive branch in the case.

"Sen. Williams faced what, in my opinion, was a cruel, unreasonable, unwarranted, improper test," Cranston said. He added that he felt that Williams failed that test "and engaged in clearly improper conduct . . . . His conduct fell short of what must be the high standard for every senator." Sen. Minority Leader Robert C. Byrd (D-W.Va.) told Williams weeks ago that he should resign for the good of the party. Williams refused, and he has brushed aside such suggestions ever since.

As Cranston spoke, Williams, 62, sat one row behind him in the Senate chamber, his brow furrowed. The two liberal Democrats are long-time Senate allies.

Williams, a 23-year Senate veteran, was convicted of agreeing to use his office in exchange for a hidden share in a titanium mining venture that was to have been financed by a $100 million loan from an undercover agent posing as an Arab sheik. He was sentenced to three years in prison and a $50,000 fine.

That conviction, Cranston said, created the momentum that has "driven the Senate toward expulsion." If the Senate bows to that momentum, he said, it would be a signal that the Justice Department "can engage in illegal activities against members of the United States Senate and get away with it."

Williams' government pension will be the same whether he resigns or is censured or expelled, according to Secretary of the Senate William F. Hildenbrand, the same as that of any senator with similar service.

This means that even if expelled and jailed Williams will be entitled to an annual pension of about $45,000, Senate floor privileges and use of the Senate gym, barbershop, library, dining room and parking lot.

The beginning of yesterday's debate was delayed so senators could attend a luncheon with President Reagan, and Williams and three others could go to a service for the late Clifford P. Case, a former senator from New Jersey.

The debate ended about three hours later with a strongly worded attack by Eagleton, who introducd himself as "the showcase liberal on the Ethics Committee." He said Williams was "a knowing participant in this sleazy enterprise" who was not "dragooned or bludgeoned."

"He was actively engaged in the enterprise," Eagleton said. "He devoted considerable time to it. He was going to participate in the rewards."