The Senate's ornately polite facade blew apart yesterday as conservatives failed for a third time to break a filibuster on school prayer but liberals could not come up with enough votes to kill the bill.

"It may be we've created a legislative gridlock," Majority Leader Howard H. Baker Jr. (R-Tenn.) said unhappily after the session in which tempers flared. But he then scheduled a fourth and possibly final attempt to break the filibuster today.

The fight was in part on the prayer bill itself but also in large part personal, involving Sen. Jesse Helms (R-N.C.), who has now pressed pro-prayer and anti-abortion provisions on the Senate for a month as riders to a debt ceiling bill that Congress must pass before it can adjourn for the campaigns and election.

Item: Sen. Dale Bumpers (D-Ark.) mistakenly referred to North Carolinian Helms as "the senator from South Carolina."

"He moved me across the line," Helms complained.

"I apologize to the other state," Bumpers shot back.

Item: Bumpers accused Helms of pushing for votes on the school prayer issue to help a direct-mail firm Helms controls raise money from conservatives even if it means keeping the Senate from more important business.

"His presses are running, his letters are going out. He is going to go after all the troglodytes in the Senate who are opposed to school prayer," Bumpers shouted.

"But how many times do we have to vote on this? What advantage is there to being on record 10 times instead of nine? The die is cast," Bumpers said.

"I understand the senator is against prayer in the schools," Helms retorted.

"Nonsense," Bumpers replied.

The motion to table and in effect kill the Helms bill was made by Sen. Barry Goldwater (R-Ariz.), the 1964 Republican presidential nominee and for a generation the "conscience of the conservatives."

Helms, Goldwater said afterward, has not "helped the conservative cause one bit," adding that he did not think even "a liberal like Teddy Kennedy" would have used tactics of the sort Helms has used in pushing for his New Right social agenda.

"This is my 48th wedding anniversary and my wife is madder than hell that I'm not there with her in Arizona," Goldwater said, plainly miffed. "I've got to be here doing what Jesse wants to do."

Helms, for his part, accused his opponents of grandstanding; he seemed especially irked by some senators who had declared they had beaten the "radical right" in the four weeks of debate.

"I didn't get up and shout the radical left has been beaten," he said. "I have sat silent in the face of all sorts of abuse directed at the people who want prayer in the public schools."

Helms' school prayer amendment would keep federal courts from overturning state laws allowing voluntary prayer in public schools, which the Supreme Court banned in 1962. Opponents call it a court-stripping bill, saying it would deprive the judiciary of its constitutional powers.

In two separate votes, liberals yesterday showed they have enough strength to block an up-or-down vote on the measure, but not enough to kill it. The vote on ending the filibuster was 54 to 46, six short of the 60 required to limit debate. Conservatives picked up only one more vote, that of Sen. John C. Stennis (D-Miss.), than they had on Tuesday when a similar debate-limitation motion failed.

When the tally was announced, Sen. Daniel Patrick Moynihan (D-N.Y.), a filibuster leader, shouted: "This was an emphatic and final rejection of the New Right. May we now go on to the nation's business, the Constitution intact, our purpose clear."

Goldwater then offered his motion to table. But while last week the Senate tabled a Helms anti-abortion rider after he failed to win cloture, this time the tabling motion failed, 53 to 47.

Virginia Sens. Harry F. Byrd Jr. (Ind.) and John W. Warner (R) voted for cloture, the legislative term for limiting debate, and against the tabling motion. Maryland Sens. Paul S. Sarbanes (D) and Charles McC. Mathias Jr. (R) voted opposite the Virginians both times.

Republican leader Baker, under intense pressure to end the prayer impasse, said that following a fourth cloture vote today he will propose that the debt ceiling bill be stripped of all amendments. That would allow the Senate to take a procedural vote without the political implications of deciding directly on the sensitive prayer issue.

The filibuster against school prayer and Helms-sponsored anti-abortion legislation has been going since Aug. 16 with only a Labor Day recess and occasional pieces of "must pass" legislation interrupting it.

Until yesterday, it had been a decidedly gentlemanly affair. Helms and his allies have rarely even participated in the debate. Sen. John P. East (R-N.C.) took the floor early in the day to defend Helms and accuse liberals of keeping the views of a majority of Americans, who he said favor school prayer, from becoming law.

"There is too much elitism in Washington today," East said. "The theory is you can't trust the local school board . . . , the American people to make a decision."