A year ago, the Senate balked for months at a Democratic proposal to hold military spending growth to 5 percent above inflation. The Republican majority relented only when Congress was about to adjourn without passing either a budget or a defense authorization bill.

That was last year. Now, the Senate is backing away from a 3 percent above inflation increase negotiated by its leaders with the White House and has cast one vote to increase defense spending only enough to offset higher prices, providing no "real" growth for the Pentagon in fiscal 1986. "I don't believe it," said Sen. Bill Bradley (D-N.J.) after the Senate voted narrowly but without much wavering Wednesday to slow President Reagan's costly military buildup.

"Last year the debate was: Can we survive with 5 percent?" Bradley said. "Now the debate is can we survive without a freeze?"

The current debate involves how much defense spending should increase, with relatively little difference in dollar amounts expected for the next year or so but with larger savings possible later. Increases over the past four years guarantee continued spending at high levels, especially for multibillion-dollar weapons systems for which only modest down payments have been made.

Bradley's comment reflects a dramatic shift in Congress' sense of priorities, reinforced by mounting frustration over reports of spending excesses by the Pentagon and defense contractors, the "fairness" issue and the Reagan administration's lame-duck problems on Capitol Hill.

In the Senate, Majority Leader Robert J. Dole (R-Kan.) and other leaders have made reduction of $200 billion-plus annual deficits the top priority, apparently with support of rank-and-file lawmakers of both parties.

Budget deficits, which have more than tripled since Reagan took office and will have doubled the national debt by midway through his second term, always have concerned Congress more than the White House, which until recently tended to suggest that economic growth would take care of the problem.

Deficits have now become a congressional obsession, made all the more compelling in recent weeks by signs of an economic slowdown that could only be exacerbated by deficits' continuing drain on investment resources and upward pressure on interest rates.

At the same time, lawmakers who used to hear from constituents about the importance of rebuilding military strength are now hearing about $640 toilet-cover assemblies for military aircraft, about low taxes and high profits for defense contractors and about weapons that do not work.

Pressure for restraint in defense spending has been building for several years in Congress. But the Senate, in particular, had held out for most of what Reagan wanted -- until last week.

"The whole climate has changed," said a senatorial aide who has sounded out opinions on defense and other issues among his boss' constituents. "The concept behind SDI the space-based Strategic Defense Initiative may be too complex for most people to understand, but profiteering at taxpayers' expense gets to them real quick," the aide added.

Others say there is a sense that international tensions have subsided, especially with renewed arms control talks with the Soviets and a return to preoccupation with domestic concerns.

And, with prosperity still the rule, despite pockets of persistent recession and signs of possible troubles ahead, it is hard for many people to understand why Amtrak, small-business loans and farm subsidies should be sacrificed to reduce deficits, especially as money contines to pour into the Pentagon.

This gets to the "fairness" issue, an especially sensitive one for Senate Republicans as they seek to retain 22 of the 34 seats to be contested in 1986. Many of them say voters might accept even the most controversial of domestic spending cuts, such as limits on cost-of-living increases for Social Security benefits, but only if other programs, primarily defense, take their share of cuts.

There is also an impression, although there are grounds to question it, that Reagan's lame-duck status is hindering the administration, at least on the most difficult issues. It remains untested in part on the defense spending issue because, although Defense Secretary Caspar W. Weinberger lobbied hard on the issue, Reagan did little personal arm-twisting.

Under the Senate-approved plan for zero real growth next year followed by modest increases in the following two years, outlays for defense would grow from $252 billion this year to $273.1 billion next year and $313 billion by fiscal 1988. Even with the real-growth freeze, fiscal 1986 outlays would grow by $21 billion over current spending.

Significantly, so great are the built-in costs that the difference between zero growth and 3 percent growth after inflation next year is only about $3 billion. The difference between the plan adopted by the Senate and the one it rejected (3 percent in each of the next three years) is $6.3 billion in fiscal 1987 and $8.4 billion in fiscal 1988.

As the Senate Budget Committee noted in recommending zero growth, which was later raised to 3 percent after inflation in a compromise negotiated between GOP leaders and the White House, the average annual increase in defense spending between fiscal 1982 and 1986 would still be 6 percent after inflation, even with zero growth in 1986. This, the committee noted, is well above this country's 3 percent real-growth commitment to the North Atlantic Treaty Organization.

In the past four years, defense appropriations have grown 35 percent in inflation-adjusted terms, or 15 percent more rapidly than in 1968 at the height of the Vietnam war. Under the Senate-approved formula, defense spending would resume growing in fiscal 1987 and 1988, rising 3 percent above inflation in each of those years.

What is disturbing many defense advocates is that the Republican Senate will be starting negotiations with the Democratic House at a low bargaining position for military spending. The House Budget Committee is considering freezing defense spending authority at current levels, with no allowance for inflation. Any compromise between the Senate and House budget panel positions would mean a cutback in real-dollar terms for the military.

Concern over the final outcome is what kept Senate Republican leaders last year from compromising early on defense. Now they had to compromise early to satisfy Republican senators, a dozen of whom joined most Democrats in the 51-to-48 vote rejecting the White House compromise on defense.

It is not all bad news for the Pentagon, however. A report by the minority staff of the Senate Armed Services Committee concluded recently that no weapons system would have to be canceled -- and many could be increased "significantly" -- under the no-growth plan adopted by the Senate.