Japan will agree to limit the number of color television sets it exports to the United States to 1.75 million a year for the next three years, government sources said yesterday.

Japanese and U.S. officials are expected to sign an agreement to that effect Friday in Washington. The level of Japanese exports would be substantially below the 2.8 million sets Japan sold in the United States last year, but above the export levels before 1976.

Officials familiar with the negotiations said the pact is also designed to encourage Japanese manufacturers to set up production facilities in the United States rather than ship complete or nearly complete TV sets to this country.

The Carter administration has been trying to balance an increasingly protectionist mood in Congress, industry and labor against the need to avoid retaliatory trade restraints by the nation's major trading partners.

The U.S. International Trade Commission ruled last March that the domestic television industry was being hurt by imports and recommended that President Carter boost tariffs on TV imports from 5 per cent to 25 per cent for the next two years.

Carter instead said he would try to negotiate voluntry quotas with Japan, which accounts for about 80 per cent of the nation's television imports.

Carter used the same approach to limit shoe imports. On Monday, administration officials said they had reached a tentative accord with South Korea and Taiwan that would limit to a total of 155 million pair - down from last year's level of 200 million - the numbers of shoes these two Asian countries could send the United States for the next four years.

The television compact apparently would limit Japanese exports of completed and nearly completed sets to about 1.56 million a year and partially completed sets - called sub-assmblies - to 190,000 a year.

Labor sources said that getting the Japanese to agree to an export figure below 2 million was a major victory for Carter's special trade representative, Robert S. Strauss, who conducted the negotiations in Washington and Tokyo. Nonetheless, sources said that labor would find it hard to acquiesce to imports as high as 1.75 million sets.

Until 1976, Japanese-made sets accounted for about 14 per cent of the U.S. market. At 1.75 million units, Japan would account for almost 22 per cent of the market if 8 million sets are sold this year.

"Until we see the language it is awfully hard to comment," one labor officials said of the agreement, but he said it is hard to think labor would be "happy."

Nevertheless, to undo the President's action and impose the Trade Commission recommendation of higher tariffs would take a congressional override.

Government and industry analysts estimate the higher tariffs advocated by the Trade Commission, set up to administer the 1974 trade law, would add between $200 million and $300 million annually to the nation's $2 billion color television bill.

The Trade Commission had recommended that restraints be placed on both color and black-and-white imports, but the President chose to negotiate voluntary quotas for color sets alone.

There was no estimate available yesterday of how many jobs if any, the U.S.-Japan agreement will add to the U.S. television industry. But on Monday a coalition of labor and industry groups formed to fight color television imports, called the Committee to Preserve American Color Television, said that a quota level of 1.3 million sets, about the import levels of the early 1970s, would add 9,000 jobs to the American color television industry.

In order to encourage Japanese manufacturers to set up plants in the United States - and employ American workers - the agreement is said to contain provisions that exempt "sub-assemblies" from the quota if about half of the total labor needed to make the complete set is provided by American workers.

According to Commerce Department figures, the United States imported 1.3 million color sets in 1972, 1.5 million in 1973 and 1.2 million in 1975. Color television imports rose to 2.8 million sets last year. In addition, the Commerce Department said about 600,000 partially assembled sets were imported in the last six months of 1977. The department did not keep figures on partial assemblies before last year.

Carter must make the final decision on whether the United States will agree to the quota level negotiated between Strauss and Japanese trade interests. When the agreement was presented Monday to the economic policy group - which is comprised of top administration economic officials - there was little opposition, although sources said Coucil of Economic Advisers Chairman Charles L. Schultze was displease with restricting trade.