U.S. trade protection reduces Japan's exports here "by about the same amount that Japanese protection reduces U.S. exports," according to a new study of American-Japanese economic problems by the Institute of International Economics.

The study, by C. Fred Bergsten, director, and William R. Cline, a senior fellow of the institute, concludes that even if Japan dropped "all overt and intangible" barriers to trade, its trade surplus with the United States -- which may hit $50 billion this year -- would be trimmed by no more than $5 billion to $6 billion.

They concluded in the detailed report that "about $5 billion worth of Japanese exports are being restrained" by various U.S. quotas and regulations -- over $4 billion of that in autos, steel and textiles.

"Although both sets of estimates must be considered quite rough, they suggest that approximate reciprocity already exists," the report said.

It rejected as exaggerated an internal Commerce Department estimate that U.S. exports to Japan could increase $17 billion if all trade barriers were eliminated.

The report's assertion that the two economies are about equally protectionist was sharply challenged "as a whitewash for the Japanese system" by Gaza Feketekuty, senior assistant U.S. trade representative. Feketekuty, addressing Bergsten, said: "I fear your paper . . . could reduce the incentive for the Japanese to make reforms and get rid of protectionism."

Feketekuty's criticism of the report was echoed by Commerce Department trade negotiator Clyde Prestowitz, who said that the American and Japanese concepts of market "openness" do not match. Prestowitz acknowledged that "protectionism in Japan has decreased, but there still is a high degree of it."

Bergsten denied that "the report is, or was intended to be, a whitewash." He cited sections of the report that called on Japan "to take the lead in eliminating (trade) restraints since they often become flashpoints for extreme policy reactions in the United States."

The main theme of the Bergsten-Cline study is that "the overwhelming cause of the present crisis" between the two nations lies in macroeconomic policies that have created a highly overvalued dollar and the fact that U.S. economic growth has been exceeding Japan's.

"There are no grounds for blunderbuss retaliation steps, by Europe or the United States, against Japan for supposedly grossly unfair trade practices," Bergsten said. He specifically said that there is "no basis" for a proposed 20 percent import surcharge, a concept that is gaining support among Democrats and Republicans in Congress.

Bergsten and Cline concluded it would be normal, at a time when Japan and the United States are in "global equilibrium," for Japan to run an annual trade surplus of $15 to $17 billion with the United States. A key part of the Bergsten-Cline argument is that the rise in the deficit "can be explained almost fully by changes in the exchange rate and national income."

In periods of high growth, the report said, imports into the United States tend to grow three times as quickly as imports into other industrial countries. This phenomenon -- a three-fold greater U.S. "elasticity" of demand than in Japan -- means that the U.S. trade balance with Japan "will continue to deteriorate," Bergsten said.

The report also challenged the widely held view that Japan's imports of manufactured goods are abnormally low. It presented data showing that U.S. exports of manufactured goods to Japan in 1983 were $1.4 billion, compared with $850 million to West Germany, usually cited as one of the least-protected markets among industrial countries.

The report called on the United States to cut its budget deficit by at least $150 billion annually by 1988, and to actively promote lower exchange rates for the dollar. To reinforce the effort to bring the dollar down, the report called on Japan to intervene in the capital markets and to take supply-side measures to boost its own economy, thereby strengthening the yen.

Although Japan had the biggest trade surplus of any nation with the United States last year -- $36.8 billion -- when ranked according to the percentage of the total trade between the two nations it was only the fifth largest, exceeded by Romania, Taiwan, Brazil and Hong Kong, according to the report.

Sen. Max Baucus (D-Mont.), who was present at a seminar Wednesday when the report was outlined to government and industry officials, praised the report but predicted that Congress would ignore its conclusions and take action directed against Japan.

"Congress is getting close to that critical point where we are going to start enacting retaliatory measures against Japan , and I fear they won't be as constructive as they should be," Baucus said. He added that an import surcharge was a strong possibility:

"We're reacting to our constituencies, and we're on a razor's edge. Since there "is no trade policy in this administration . . . it will have to be made in Congress."