When the clock struck 1 Wednesday morning and the Senate packed it in after another long and frustrating night, Jesse Helms of North Carolina walked over to Alan K. Simpson of Wyoming and stuck out his hand.

"Let's be friends," he said to his fellow Republican, who an hour earlier had savaged Helms' unrelenting efforts to filibuster the gasoline tax bill to death as an "obdurate . . . obnoxious performance."

As several other senators and aides watched anxiously, Simpson refused to get up from his chair to seal the handshake in the accustomed fashion of this most courteous of legislative chambers. He stared at the man standing over him for several seconds without saying a word. And soon Helms turned away, the outcast in a Senate he said he dearly loved.

For 12 days and nights, Helms, the leader of the New Right, chairman of the Agriculture Committee and master of delay, took on his president, his party leadership and the vast majority of his Senate colleagues in a lonely battle against the bill to raise the tax on gasoline by a nickel a gallon.

There were several times over those days when it seemed as if he would win, but as the end approached the major casualty seemed to be none other than Helms. The Senate leadership said it will have the votes to cut off his filibuster at 9:30 this morning, and two hours later, if the leadership headcounts prove accurate, the bill will pass.

When and if that happens, the 97th Congress will be over -- a Congress that saw Helms' clout and standing in the Senate decrease, even as he wielded considerable influence nationally as the leader of a multimillion dollar conservative fund-raising and opinion-molding empire.

Though he proved adept at the filibuster, many of Helms' colleagues found him less capable as a floor leader for farm bills as chairman of Agriculture.

On one crucial occasion in 1981, Helms virtually gave over the job of shepherding the agriculture bill to passage to moderate Finance Committee Chairman Robert J. Dole (R-Kan.), second-ranking on the farm committee. Some Republicans complained that he seemed interested only in the major crops in his state, tobacco and peanuts.

On the Senate floor Tuesday night, Simpson, when blasting Helms for his filibuster, said: "It seems the whole issue of the senator's tenure, as I have observed it in my short term, seems to be 'How is it playing in North Carolina as to peanuts, tobacco and family farms?' "

And Sen. Ernest F. Hollings (D-S.C.), also from a tobacco state, recently complained that Helms was the industry's worst enemy. "Old Jesse almost did us in by making Sen. Thomas F. Eagleton D-Mo. so mad that every time a 'bacca bill would come up Eagleton would be down in the well just waitin' for it."

The turning point in the enervating test of will on the gasoline tax this week came shortly before midnight Tuesday, after Helms had vowed to keep his colleagues in Washington over Christmas if that was what it took.

The Senate Democrats--some of whom opposed the bill, many of whom had simply run out of patience as Majority Leader Howard H. Baker Jr. (R-Tenn.) maneuvered with Helms -- held a heated caucus and reluctantly agreed to stick around and support one final vote to end the filibuster.

"We ought to have a showdown or we ought to go home," Minority Leader Robert C. Byrd (D-W.Va.) told Baker. "Don't run us through this mill much longer."

Baker took the cue and filed the cloture petition that is forcing this morning's vote. "It was getting real tight with Byrd and the Democrats before their caucus," said a Republican leadership aide. "But when they decided to stick by us one last time we knew the fight was almost over. That allowed us to show Sen. Helms we would take him to the nth degree, just as he had done to us."

After the cloture petition was filed, Helms persisted in stringing out the Senate's lame-duck session as long as possible. Majority Whip Ted Stevens (R-Alaska) pleaded with Helms to agree to have the vote yesterday instead of today, saying it was "the finest Christmas present the senior senator from North Carolina could give us." But Helms said he needed an extra day to read the conference report on the gasoline tax bill.

"I apologize to my colleagues for the inconvenience," he said. "But I really should not be the one making the apologies. The president should apologize, the people who pushed this measure on us during a lame-duck session should make the apologies . . . . This is a bad bill coming at a bad time."

When Baker presented President Reagan with a list of senators to telephone yesterday to lobby for the bill, Helms' name was not on it. But Reagan had called Helms last week, urging him to drop his filibuster, and early yesterday morning the senator recalled their conversation:

"I reminded the president that he once said it would take a palace coup for him to support a tax bill like this. 'Mr. President,' I said, 'when did the palace coup occur?' Then I went down the provisions in the bill with him. 'Do you like this?' I asked. And he said, 'No.' 'Do you like that?' I asked. And he said, 'No.' "

Several senators lamented this week that the rift over the gasoline tax would do lasting harm to the Senate and to the relationship between the Helms faction and the Republican leadership. But both Baker and Helms dismissed such talk yesterday.

Baker said: "I don't think there's much damage done. We've been through a trial by fire, a trial by ordeal . . . . But a coalition of senators on both sides of the aisle held together. Instead of more divisiveness and fractiousness next year, I think the contrary will be true."

"After Christmas, after we get a little rest," Helms responded, "I think we'll work together."

"I don't know how much damage was done," said one senator who witnessed Simpson's rebuff of Helms. "But I can tell you it was pretty tight. I'd never seen anything like what happened last night around here before. I think it will take more than the Christmas spirit to repair things.