House and Senate budget leaders warned yesterday that Congress' six-year-old experiment with spending self-discipline is in danger of collapse as they groped for a way out of their deadlock over a budget for 1981.

Stepping up pressure for a compromise, Rep. Robert N. Giaimo (D-Conn.) and Sen. Ernest F. Hollings (D-S.C.), chairmen of the two Budget committees, indicated they will try to block all but the most urgent spending bills until the budget standoff is broken.

Giaimo also prodded his colleagues with a warning that continued delay jeopardizes Congress' claim to be proposing the first balanced budget in 12 years.

Unless Congress can count on savings anticipated in the budget plan, Giaimo said, he will be forced to try to add up to $10.6 billion to the budget totals, thereby forcing Congress to acknowledge a deficit or to cut spending programs or raise taxes.

But the prospects of an immediate break in the impasse appeared uncertain, as a tentative compromising proposal from the House Budget Committee got a chilly response from the Senate.

"The budget process is in terrible jeopardy," said Hollings, noting that the Senate Budget Committee adamantly rejected overtures from its House counterpart to shift some money from defense to domestic programs.

"We got a mess on our hands," if a compromise isn't reached soon, said Giaimo.

"I'm not encouraged at all," said House committee in trying to fashion O'Neill Jr. (D-Mass.) as he joined the House Committee in trying to fashion a compromise that would satisfy both the defense-minded Senate and the Democratic majority in the House that wants more spending for social programs.

The budget's current problems began last week when the House rejected a defense-heavy $613.3 billion budget plan that had been drafted in a House-Senate conference and then, after many Democrats had left, voted to stick by the plan's proposed outlay of $153.7 billion for defense.

This seemingly contradictory stance strengthened the hand of defense advocates in the Senate and left the House committee with little maneuvering room.

The committee's tentative offer is to shift $300 million from defense to transportation and income security, including funds to meet O'Neill's demand for more money for fuel assistance to poor people -- along with a $2 billion cut in the $171.3 billion figure for long-term defense commitments, principally affecting military hardware.

House liberals are contending that this defense figure, involving funds that are obligated but not necessarily spent during a budget year, is too high and must be scaled back so domestic programs won't be squeezed out to future years because of prior military commitments.

At the end of a two-hour meeting with O'Neill, Giaimo said the House and Senate may return to conference on the budget today, apparently depending on the success of further informal contacts with the Senate conferees.

But, pending a settlement, he said he would try to prevent a vote on all but "absolutely essential" spending proposals for the rest of 1980, such as special unemployment benefits that are expected to run out later this week. A total of $14.3 billion in spending proposals has been frozen until the budget is approved, requiring a special waiver if the proposals are to be acted on sooner. Hollings also said he is inclined to oppose any budget waivers.