Now that the initial furor raised here over the alleged pirating of computer secrets from International Business Machines Corp. by Japanese businessmen has died away, the Japanese public appears to have been left with an impression that the United States intended the disclosure to help damage Japan as an industrial rival.

The incident has also intensified concern here over what the Japanese perceive as a growing anti-Japanese mood in the United States. It featured a 10-month "sting" operation by FBI undercover agents, and last week 21 persons, mainly Japanese, were charged with conspiracy to steal proprietary technical data from IBM.

The Japanese have reacted negatively to the methods used by the FBI in exposing the case, which, according to widespread speculation here, was put together with the help of IBM to unfairly discredit Japan's trade practices.

Undercover operations are not normally employed in Japan. The press here has compared the FBI's tactics to those it used in the so-called Abscam cases, in which seven U.S. congressmen were convicted of taking bribes from agents posing as Arab sheiks.

Should Hitachi and Mitsubishi Electric, the two companies implicated in the case, be forced to defend themselves in U.S. courts, reliable sources here said, their lawyers would almost certainly attack the FBI's tactics.

A grand jury in San Jose, Calif., reportedly will decide sometime next week whether it will hand up indictments.

Hitachi and Mitsubishi have staunchly denied any wrongdoing and have broadly challenged FBI allegations in preparation for possible lengthy legal battles.

White House officials and U.S. attorneys have denied, in effect, that the charges were intended to put pressure on Japan in trade negotiations in which American officials are strongly urging the Japanese to broaden their markets for foreign goods.

Reports in the Japanese press have increasingly highlighted what has been portrayed as a groundswell of anti-Japanese sentiment in the United States. Many Japanese believe it stems from a mistaken American perception that Japan's huge trade surpluses result from unfair trading practices.

In public statements, officials in Tokyo and Washington appear to have tried to downplay the computer case to reduce the risks of any serious political difficulties between the two countries.

Thoughtful Japanese have warned repeatedly against an emotional reaction on either side that could unsettle the relations between the two countries.

Many Japanese have expressed concern that their image in the United States may suffer. In a dispatch from Washington, the Yomiuri Shimbun, a major Japanese daily, said: "Whatever the truth of the matter, the worst thing is that it may strengthen the American public's view that Japan is a 'sneaky' country."

The Asahi Shimbun, another major daily, said the incident had created, on the Japanese side, the impression that Japan had been the victim of an entrapment plot. On the American side, it said, it had reinforced the image of the Japanese as a people "who think they can solve all their problems with money."

The incident comes amid rapidly increasing competition between Japan and the United States for the upper hand in the area of highly sophisticated electronic technology. While the Japanese have acknowledged that the United States maintains the edge in many of these fields, some industry observers here have asserted that the FBI's disclosure was intended to blunt Japan's competitive drive.

Japanese companies like Hitachi have moved into a dominant position in the world market for computer memory chips. The six major Japanese computer makers, however, together have only about one-third of IBM's total global sales and lag far behind the American giant in the development of software programs for computers.

A common view in Japanese industry circles is that American manufacturers have often tried to shield themselves by tying up Japanese competitors in lengthy and costly legal actions.

Japanese executives have interpreted the industrial espionage charges brought by IBM as another possible example of such tactics.

Japanese government officials, meanwhile, privately expressed dismay at the timing of the FBI disclosure. It came only a few days after the Reagan administration announced it had decided to ban the export of highly sophisticated exploration and assaying technology for use in a multimillion-dollar, Japanese-Soviet petroleum project.

The Japanese already have sunk $220 million into the Sakhalin Island project north of Japan, and Washington's imposition of the ban has upset government and business circles here and called into question the viability of the venture.

In late May, Tokyo announced a second group of steps to reduce its trade barriers and open its markets more widely to foreign goods. Although the measures fell short of American trade officials' expectations, Japanese leaders emphasized that they had taken them in the face of heated opposition from powerful domestic interest groups--and at considerable political cost.

Officials have suggested that the computer charges have added to the public's impression of coldheartedness on the part of Japan's U.S. ally. While they say a serious outbreak of anti-American sentiment here is unlikely, officials have warned that resentment of the United States could, over time, reduce Japan's flexibility in managing its U.S. ties.