Top administration economic advisers will urge President Reagan today to press Prime Minister Yasuhiro Nakasone to reduce Japan's burgeoning trade surplus with the United States and to leave open the possibility of retaliatory action if he fails, administration sources reported.

The members of the Cabinet-level Senior Interagency Group on International Economic Policy (SIGIEP) split, however, over whether Nakasone should be asked during his Jan. 2 California meeting with Reagan to set actual year-by-year targets for increasing Japan's imports of manufactured goods.

The results of discussions during five Cabinet-level sessions over the past 10 days will be sent to the president today at a National Security Council meeting to develop administration strategy for the Nakasone talks.

The discussions were unusually intense, administration sources said, reflecting a growing frustration within the upper echelons of government with the failure to make significant inroads for American products into the Japanese markets and the belief that a new, tougher stance on trade with Japan is needed.

Top administration advisers are concerned that the failure to achieve results in past talks has contributed to what Commerce Undersecretary Lionel H. Olmer calls a "grotesque" U.S. trade deficit -- expected to total a record $130 billion for 1984 with $35 billion of it due to Japan -- that is damaging relations between the United States and Japan, its closest Pacific ally.

They are also worried that the brief time Reagan and Nakasone will meet -- they will spend only about 2 1/2 hours together at the president's Santa Barbara ranch, including a working lunch -- will allow the Japanese prime minister to escape tough talk on trade.

The two leaders developed a close "Ron and Yasu" relationship during four previous meetings, and some administration officials have characterized the discussions within the government as "a battle for the heart and mind of the president" to convince him to emphasize to Nakasone the importance of Japan's reducing its ballooning trade surplus with the rest of the world as well as the United States.

Administration officials compared Japan's current account surplus -- the broadest measure of its international transactions -- with the rest of the world with the huge surpluses that oil exporting nations piled up during the height of their boom years.

If the dangerous aspects of Japan's trade surplus do not arise in the Reagan-Nakasone talks, leading administration officials said, it will be more difficult for U.S. Trade Representative William E. Brock and Commerce Secretary Malcolm Baldrige to convince their Japanese counterparts that the Reagan administration is seriously concerned about the issue.

As a result, the tenor of discussion within the administration has changed, government officials said. "No one argues any more that pressing Japan on trade would erode our ability to have them act on defense or the Mideast or some other area," said one Japan expert in the government.

Even Treasury Secretary Donald T. Regan switched to push for a new, tougher line on trade with Japan, officials said, and the State Department agreed that new moves were needed, but cautioned against using too much pressure. The Pentagon has not taken a major part in the discussion so far, but traditionally has argued that pressure on the trade front could harm U. S. use of military bases in Japan.

The recommendation to the president calls for a quick start to high level -- undersecretary or above -- talks that will lead to immediate opening of Japan to increased American imports of manufactured goods in areas where the United States is considered competitive and where there is a Japanese need for the products.

If these talks fail to bring a quick increase in U.S. sales, the United States would retaliate against Japanese exports to the United States. If U.S. companies, for instance, are not allowed to sell more sophisticated telecommunications equipment, Japanese exports would face either presidentially initiated trade complaints or more subtle administrative moves such as the rejection by the Federal Communications Commission of foreign test data on telephones, sources said.