Japan and China are set to sign a nuclear cooperation agreement that will open the door to a Japanese role in China's program to build about 10 nuclear power plants by the year 2000 at a cost of up to $20 billion, Japanese officials say.

The accord, providing safeguards against the use of transferred technology for military purposes, is similar to the one signed earlier this week between China and the United States during the visit of Chinese President Li Xiannian to Washington. The timing is said here to be coincidental.

Japan is a world leader in nuclear power, with 30 plants in operation accounting for 20 percent of the country's total generating capacity. It has also sold generating equipment used in foreign nuclear plants but, until now, has never sold reactors.

Mitsubishi Heavy Industries already has signed a contract to provide a pressure vessel, a key reactor component, to a Chinese plant being built at Qinshan. The vessel is to be shipped from Japan next January.

"After formal signature of the agreement, we plan to start full-scale efforts" for more Chinese nuclear contracts, a Mitsubishi spokesman said today. The signing is to take place at a ministerial-level economic conference scheduled for Tokyo on July 30-31.

However, Chinese displeasure with Tokyo's mounting trade surplus with Peking could put a brake on nuclear sales. Japan recorded a $1.3 billion surplus in two-way trade of $13.2 billion in 1984. In the first six months this year, Japan's surplus grew to $2.8 billion in total trade of $9.2 billion.

Japanese capital and consumer goods are playing a crucial role in China's ongoing modernization programs. But China, angered by Japanese traders' claims that they cannot find more to buy there, is applying pressure.

This year, China virtually has stopped signing new contracts for many Japanse product lines, leading to predictions here of a major downturn in trade in the year's second half. Correcting the imbalance will be a major agenda item at the Tokyo meeting later this month.

To date, Western European companies have held the inside track in China's plant construction program. Japanese participation was hampered by lack of a nuclear agreement, so Peking and China began negotiating one in 1983. It was initialed in Peking earlier this month.

There already has been some nuclear cooperation, however. In addition to the Mitsubishi sale, a Japanese company signed a contract last year to conduct a preliminary survey of uranium reserves in China's Yunnan Province.

Japan already has bilateral nuclear cooperation agreements with the United States, Canada, Australia, France and Britain. Nuclear trade with those countries, however, consists primarily of purchase of fuel and equipment by Japan. Japan also has agreed to give technical aid to South Korea's nuclear power program.