The administration is "worried" because congressional efforts to cut the deficit seem to be bogging down and "time is wasting" as senators from both parties balk at President Reagan's proposed domestic spending cuts, a senior White House official said yesterday.

"They're shy so many billions at this point, refusing to bite the bullet, not showing any desire to want to be statesmanlike," said the official, who was interviewed on condition that he not be identified.

The official said the slow progress could "complicate" but "doesn't negate" the administration's agenda for tax simplification this year because the White House had hoped to win the budget battles first, before turning to overhauling the tax code.

His comments came the day after the Republican-controlled Senate Budget Committee deadlocked over Social Security and rejected Reagan's proposals to end subsidies for Amtrak, sharply restrict student loans and wipe out or drastically reduce a variety of other domestic spending programs.

Many lawmakers have blamed Reagan for the sputtering drive to reduce the deficit, saying his rigid stands on defense spending and Social Security cost-of-living increases have made it difficult, if not impossible, to put together a consensus on domestic spending cuts.

But the White House official said Congress was to blame for the slippage.

"It's nothing that we've caused," the official said. "They've caused it themselves among themselves. Not anything we've said. They brought up the farm bill and made that the first thing out and spent 10 days or so on that."

Separately, White House spokesman Larry Speakes said the Senate Budget Committee was "marching in the wrong direction" by failing to come up with the spending cuts Reagan wants. "We are deeply disappointed with the . . . committee's failure to come to grips with the deficit," Speakes said.

Speakes added that the committee so far has "only come up with about a third of the cuts necessary to get to their goal of fifty to sixty billion in deficit reductions."

The committee votes last week were the first step in producing a budget resolution which sets overall targets. Later, Congress approves specific spending bills. Committee Chairman Pete V. Domenici (R-N.M.) said the panel will try to break the impasse on Social Security next week. It rejected proposals Thursday to eliminate Social Security cost-of-living increases for a year, as well as rival proposals to leave the increases intact.

The senior White House official said the looming 1986 Senate races have figured in part in the fractured deficit-cutting efforts because GOP senators appear to be distancing themselves from Reagan's budget proposals. For example, he said, Sen. Robert W. Kasten Jr. (R-(Wis.), who is up for reelection next year, voted with the administration only 30 percent of the time in the budget committee.

"One of the things we're discussing and will be thinking about next week is how do we attack that problem?" he said.

"We have not really been confrontational on the budget yet," the official said. Asked if that would come later, he said, "It's the old recipe for rabbit stew. First you catch the rabbit. What the hell is their budget? How can you be confrontational until you find out what their budget is? We've got to find out what their budget is and then see how many votes will it have? And then see if it's worth it."