In a preliminary decision that has major implications for U.S.-Japan trade relations, the International Trade Commission ruled yesterday that American semiconductor makers are being hurt by Japanese sales.

The unanimous finding by the quasi-judicial ITC came in the first investigation President Reagan has ordered into possible unfair trade tactics by Japan in the competitive high-technology industry.

Memory chips are a key component in computers and telecommunications systems. A major U.S. presence in the field is considered necessary to maintain America's position as a leader in high-technology innovation. Although the United States once was the world leader in semiconducters, Japan began dominating the market in the late 1970s.

The Japanese were accused of gaining more than three-fourths of the American market for dynamic random action memory chips (D-RAMs) by dumping the semiconductors at less than their production costs. As a result, prices last year fell to less than $2 for a chip that would have cost between $18 to $20 in 1984, the Commerce Department said. That drop cost the domestic industry about $900 million, the agency said. Imports more than doubled between 1984 and 1985, to 25 million chips, while their value dropped.

The president's decision to initiate the investigation shook Japan, the United States' major Pacific ally, whose officials deny they used unfair tactics to get into the American market.

The case is considered the most sensitive of four filed this fall and winter against Japanese chip makers because it is the only one begun by the government and is part of President Reagan's program to try to reduce America's record trade deficits. In initiating the complaint, Reagan followed the recommendation of a new trade strike force headed by Commerce Secretary Malcolm Baldrige.

Yesterday's ITC ruling is the first step in a long road to assessing whether penalty duties should be imposed on Japanese semiconductor imports. Baldrige said the penalties could include higher tariffs for the next generation of chips, the planned one-megabit, which is capable of storing more than 1 million pieces of information.

The next significant step comes in late May, when the Commerce Department has to decide whether Japan in fact is selling D-RAM chips in the United States at a price less than the cost of their production -- the legal definition of dumping.

"Indications are that Japan's costs of production are substantially above the current selling price," Baldrige said last December in announcing the complaint.

The May decision will come shortly after the economic summit of leading industrial nations in Tokyo. The government of Prime Minister Yasohiro Nakasone already has indicated concern that it will be embarrassed at home by accusations that its massive trade and balance of payment surpluses bear some responsiblity for the world's economic woes.

The Japanese reopened talks here this week with U.S. trade officials in an effort to settle the investigation, as well as the other complaints brought by domestic chip makers.

In the major complaint brought by the U.S. industry against Japan, the Semiconductor Industry Association charged that its members were denied full access to sell in Japan.