Congress hopes to break a deadlock and pass the Social Security bill on Thursday and then quit for the year, leaving final action on energy legislation for 1978.

House and Senate conferees reached agreement last Friday on a payroll tax increase that would bring in $227 billion over the next decade to shore up the Social Security trust funds. But they broke up in disagreement over a nongermane Senate amendment for a $250 annual college tuition tax credit.

Over the weekend the administration denounced the holdup, blaming Sen. William V. Roth Jr. (R-Del.), the tax credit's leading sponsor among the conferees. No mention was made of Senate Finance Committee Chairman Russell B. Long (D-La.) who stood solidby beside Both in insisting on the credit.

Yesterday at least one Democratic Senate conferee reportedly told Long he should stop supporting Republican Roth the pass the Social Security Bill. Leading House members were also trying to work out a compromise with Long. There was a growing feeling the Social Security bill will be passed before Congress goes home for Christmas.

But there isn't time to pass President Carter's omnibus energy bill this year. The best that conferees can do is work out an agreement on natural gas and taxes, and time is running out on that. Congress plans to adjourn until Jan. 19.

At the White House, meanwhile, a spokesman acknowleged that the President's energy program stands no chance of passage in the remaining legislative days of the session. The White House even expressed doubts that a general agreement could be worked out by the congressional conferees that might serve as a basis for early action on the program in January. (Details, A2.)

The Senate Finance Committee plans to caucus this morning to rethink its position on Social Security, energy taxes and a tax on coal to finance benefits to coal miner victims of black lung disease. It has been butting heads with House conferees on all three.

Carter's energy bill is hung up in conference principally on the issues of natural gas pricing and his proposed on domestic crude oil. The House sided with Carter on both while the Senate opposed him on both.

Long would support the crude oil tax if some arrangement is made to give more money to the oil industry, either as production incentives or a price increase. The tougher issue appears to be natural gas. Carter would continue price controls at a higher level than now and extend federal price regulation to the intrastate market. The Senate voted to deregulate newly discovered natual gas after two years. Deregulation is worth billions of dollars to the oil-gas industry.

The nontax conference met for the sixth day yesterday on the gas issue without making any progress. The Senate conferees are split 9 to 9 and have been unable to agree on a compromise to offer the House. Their proposal last Thursday to recess the public conference to give a chance for private talks produced nothing.

Senate conferees told their House counterparts yesterday that if they are ever going to reach agreement the House must take the first step to get the process started.

"I don't know how much longer grown people can sit here and listen to us," said Sen. Pete Domenici (R-N.M.) after an hour of speeches accomplished nothing.

The conference recessed for more private talks. Several House conferees who support the President me tyesterday afternoon and said they will offere the Senate a proposal today.It was described as "a far cry from deregulation" and almost certainly not acceptable to a majority of the Senate. But it might be the start of bouncing proposals back and forth, which is the way a conference is supposed to work.

House members are on call to return to Washington on Thursday for a final vote on Social Security and the water pollution control conference agreement. If the Social Security bill isn't ready, present plans are to put off action on the water pollution bill and close down Congress until next year.

The Senate also has before it a supplemental appropriations bill involving a fight with the House over money to complete several research models of the B-1 bomber that Carter has decided to scrap. The Senate voted to rescind previously approved research and development monty, but the House insisted on spending it. If the Senate insists on its position, the House will leave that issue until next year also.