The International Trade Commission ruled unanimously yesterday that imports are seriously damaging the domestic color television industry and will recommend to President Carter on Thursday ways to protect the industry.

American consumers spend about $2 billion a year on color televisions. Imported sets, which accounted for 17.9 per cent of sales in 1975, climbed to 42 per cent last year.

Lobbyists for domestic manufacturers and unions said they were pleased with the 6 to 0 vote yesterday and say they will be "very disappointed" if the commission recommends anything less than import quotas to President Carter.

Four of the six commissioners found that color television imports - mostly from Japan but also from Taiwan and South Korea as well as other countries - are seriously hurting the industry today and the other two found that imports were a "threat" to the industry in the future.

The commission has a variety of remedies it can recommend ranging from special loans and grants for television workers and manufacturers to tariffs or quotas. The commission can recommend combinations of relief, such as adjustment assistance and quotas.

The International Trade Commission decision was assailed by the American Retail Federation, which charged that the decision - coupled with similar decisions on shoe imports and honey imports - is "thrusting the United States into a position of trade protectionism which almost guarantees retailiation by our trading partners."

The association of retail stores said that if President Carter should adopted the trade commission's recommendations it will "cost the U.S. consumer several billion dollars and further fuel inflation."

The Council on Wage and Price Stability, the White House's inflation monitoring agency, testified before the trade commission that any move to restrict color television imports would cost American consumers between $200 million and $300 nillion a year.

Last month the trade commission recommended that President Carter impose a combination of tariffs and quotas to protect American shoe makers.

The President has 60 days to accept or reject the trade commission's recommendations - provided at least three of the six commissioners can agree on the same remedy. If he should reject them, Congress has 90 days to overrule him.

American labor unions, major backers of Carter in his bid for the presidency, have turned increasingly protectionist in recent years as they believe more and more American jobs are being lost to imports. The nation's major trading partners - Europe and Japan - are watching the shoe and now the television decision carefully to see if Carter will resist union pressures to impose quotas and tariffs.

Commissioner Will E. Leonard Jr., who voted that the black-and-white industry as well as the color industry was suffering because of imports, said that the 11 American firms who make television sets have seen their aggregate pre-tax profit margins drop from 8.7 per cent in 1971 to 0.8 per cent in 1975 after losing 1.3 per cent in 1974.

Leonard said that 8 of the 11 firms had losses in 1975, two had pre-tax profits between 0 and 5 per cent and one firm had profits between 5 and 10 per cent.

The number of persons employed in making color television sets in the United States declined from 36,500 in 1971 to 23,000 in 1975, a drop of about 37 per cent, the commission estimated. Imports of black-and-white and color sets, which totalled about 4.2 million sets in 1975, will total about 7.8 million sets this year.

An official of the Japanese embassy said that Japan had expected the trade commission would find that imports are damaging the domestic television industry. "We are anxious to follow the International Trade Commission's recommendations," he said.

Three commissioners found that both black-and-white and color sets were being damaged now - Leonard, chairman Daniel Minchew and George M. Moore. Commissioner Italo H. Ablondi found present injury only for color sets and commissioners Catherine Bedell and Joseph O. Parker found the threat of future injury.