Amid mounting calls on Capitol Hill for Japan to boost substantially its spending on defense, the Japanese Finance Ministry today announced plans to limit increases in the 1983 military budget to 5.1 percent, a figure that undershoots by a considerable margin earlier forecasts and this year's 7.6 percent spending increase.

Prime Minister Yasuhiro Nakasone's Cabinet is expected to add more funds to finance officials' $11.3 billion spending proposal before it is scheduled to formally approve the package Dec. 31. The final amount, nevertheless, is certain to fall short of U.S. demands that Japan move more quickly to bolster its defensive armor to help offset the burden of American military commitments in the Pacific.

A senior Finance Ministry official, who briefed reporters here today on the condition that he not be identified, said that the austere spending limits had been set in a bid to prune Tokyo's bulging budgetary deficits. In so doing, fiscal authorities rejected a Defense Agency request that military outlays be expanded by 7.35 percent next year.

Meanwhile, the Tariff Rate Council gave final approval today to a package of tariff cuts including levies on tobacco, chocolate and biscuits. Opposition within the ruling Liberal Democratic Party had forced further consultations between government and party leaders on the proposed cuts, which include a reduction in the tobacco tariff from 35 to 20 percent.

In a statement earlier this week, Nakasone, who has stressed the need to improve troubled ties with the United States, indicated that the defense budget should be increased by a minimum of 7 percent to honor Japan's commitments under its mutual security arrangement with the United States.

Under Japan's complex budgetary process, fiercely competing government departments are now set to vie for shares of about $250 million in extra budgetary funds set aside by the government. Final appropriations are scheduled to be hammered out in negotiations among the ministries before the Cabinet's adoption of the completed budget Friday.

According to defense and political analysts here, Nakasone's attempt to up the defense ante substantially may prove the first real test of his ability to guide the country's powerful senior bureaucrats to a decision on a key national issue. In the speculation of Japanese journalists and knowledgeable observers, 1983 military outlays were likely to fall ultimately within a range of between 6 percent and 7 percent.

Tokyo's decision on defense spending is important because it comes at a time when relations between Washington and Tokyo have been badly frayed by a huge U.S. trade deficit with Japan, expected to hit a record $20 billion this year.

In the background, a growing number of U.S. congressmen, along with elements in the American public, have charged that Japan is shirking its responsibilities by keeping a tight lid on its military spending.

Earlier this week, the U.S. Senate unanimously adopted a nonbinding resolution calling on Japan to expand its defensive capabilities and, specifically, to undertake adequate defense of its own sea lanes by 1990. Reagan administration officials have, in recent weeks, told Japanese officials behind the scenes that an increase of less than 7 percent in next year's military spending plans would meet with a chilly response in Washington, informed sources here said.

According to a senior Finance Ministry official, a 7 percent solution on defense would require that most of the government's remaining $250 million fund for its ministries be funneled into military spending. In 1982, they said, the Defense Agency received roughly one-third of a larger pool of these adjustment funds. This year, however, competition is likely to be amplified by the fact that appropriations for most government departments are slated for substantial reductions.

Defense expenditures already have been given a top priority over other government departments in the Finance Ministry's preparation of what has been dubbed in the Japanese press a "less butter, more guns" 1983 budget.

Under the ministry's draft proposal announced today, most government bureaucracies will be forced to accept less funds than this year in an overall spending program expected to amount to roughly $210 billion, up only 1.4 percent from 1982. Only outlays for defense, foreign aid, energy-related projects and social security are slated to increase on a year-to-year basis.

Observers said that, by managing to engineer an 11th-hour boost in defense spending during negotiations this week -- say between 6 percent and 7 percent -- Nakasone hopes to impress on Washington the difficulties given Japan's current fiscal constraints in a society where antiwar sentiments remain strong.

Nakasone, who came to office Nov. 26, is scheduled to go to Washington next month for talks with President Reagan that are expected to center on Japan's commitments on trade and defense.

Finance Ministry officials insisted that their scaled-down appropriations for defense would not affect Japan's ability to fulfill its current five-year procurement targets for advanced military weaponry. The reductions had been made possible, they said, because of a freeze on the wages of military personnel scheduled for next year.