U.S. and Soviet nuclear arms experts opened a special round of talks today on the unratified SALT II treaty limiting strategic weapons, which President Reagan said in May the United States would no longer honor.

The Soviets, who requested the meeting last month, were expected to use the session to criticize the U.S. declaration. American delegates were prepared to listen to the Soviets' complaints and to raise questions about alleged Soviet violations of the 1979 pact and of other arms control agreements, U.S. officials said.

The two sides are not expected to achieve significant breakthroughs at either the SALT II discussions that began today or the U.S.-Soviet meetings on nuclear test ban issues scheduled to start later this week, U.S. and West European officials said. Instead, the holding of the talks was described as a sign that the superpowers wish to improve the climate of their relationship before a possible summit meeting later this year.

President Reagan announced May 27 that the United States no longer felt obligated to respect nuclear arms limits fixed by the SALT II treaty. The Senate never ratified the accord, but both Washington and Moscow said they would respect its provisions as long as the other side abided by the treaty.

The U.S. announcement in May to abandon the treaty drew sharp criticism from U.S. allies and Congress, although the United States has stayed within the SALT II limits. The administration said it would decide later this year whether to breach the SALT II ceilings by adding to the U.S. nuclear arsenal, depending on the "nature and magnitude" of the Soviet threat irrespective of the treaty.

The Soviet Union roundly condemned the U.S. action, saying that Reagan had released the brake on the arms race. Moscow then asked in late June for a special meeting of the Standing Consultative Commission to discuss the issue.

The commission holds regular meetings each spring and autumn. It was established in 1972 by the Antiballistic Missile (ABM) Treaty to deal with disputes over compliance with nuclear arms accords.

The U.S. administration initially was skeptical of the Soviets' request for the meeting, fearing that it was just a publicity stunt. But the White House announced on July 16 that it had agreed to the meeting.

"The Soviets probably saw their call as a propaganda move to call attention to an unpopular U.S. decision. I think our answer was a skillful way to put the burden back on the Soviets, and give us the chance to press them to discuss their violations," a U.S. official said.

The official, who was not a member of the delegation here, added that the meeting could lead to progress only if the Soviets respond in new ways to the U.S. charges that the Soviets have violated the SALT II treaty.

The U.S. delegation is headed by Gen. Richard H. Ellis, former head of the Strategic Air Command. The chief Soviet delegate is Gen. Vladimir Medvedev of the Soviet General Staff.

The American delegation issued a two-paragraph communique late this afternoon announcing that two rounds of talks had been held, and saying that the United States agreed to hold the meeting "as a sign of our desire that the Soviet Union join us in establishing an interim framework of truly mutual restraint."

The communique added that U.S. acceptance of the meeting "stands in contrast to the position taken by the Soviet Union in 1983, when the United States asked for a special session to discuss compliance matters relating to the SALT II treaty."

In rejecting the U.S. request in 1983, the Soviets said the complaints raised by the United States could be handled in normal meetings of the commission.

It was understood that the talks would continue at least into next week, although a U.S. official said it was "impossible to predict" how long they would last. The commission's meetings normally run at least two weeks and are kept highly confidential.

The United States has accused the Soviet Union of violating several provisions of the SALT II treaty and other arms accords. The alleged Soviet violations were cited by the administration as a major reason for deciding to stop observing the SALT limits.

The Soviets have denied that they are in violation of SALT II and the ABM treaty.