The Soviet government announced plans yesterday for modest growth in key sectors of its economy for the next four years. It conceded serious shortfalls in 1981, but asserted that the overall goals of the current five-year plan would be reached.

Finance Minister Vasily Garbuzov, in outlining the 1982 budget, announced that defense spending would be the equivalent of $25 billion, the same as this year.

Western experts interpret this figure as an indication of the impression the Russians want to create, not of the real level of defense spending. Military allocations are believed to be hidden in various other budgetary expenditures.

Garbuzov's speech followed a warning by President Leonid Brezhnev Monday that the plan was "difficult and demanding" and that it calls for improved labor productivity, better management and thrift. Brezhnev also reported that $42 billion in planned expenditures on capital construction has been cut, presumably to be diverted for defense needs.

Garbuzov spoke at a session of the Supreme Soviet, which is to approve the 1982 budget as well as the economic plan through 1985. He said Moscow was trying to limit the burden of its military payments. "But the aggressive intentions of the West, especially the United States, run counter to this policy and the arms race has now reached unprecedented levels."

In a generally gloomy speech, Deputy Premier Nikolai Baibakov, the economic planning chief, reported that production goals for this year in agriculture, coal, steel and construction would not be met. He predicted, however, that both agriculture and industry would pick up over the remaining four years of the current plan.