West Germany's ruling Social Democratic Party rejected calls today for a freeze in NATO plans to deploy new U.S. nuclear missiles in Europe, but reserved the right to reconsider a moratorium again next year.

The outcome represented a victory for Chancellor Helmut Schmidt and party moderates, who had argued it would hamper Western arms-control aims by relieving pressure on Moscow to negotiate reductions in its arsenal of nuclear weapons aimed at Europe.

Proponents of the moratorium had tried to rally support by claiming common cause with the "freeze campaign" in the United States.

In a powerful last-minute appeal to the party's national convention, Schmidt asserted that if a freeze were effected in Europe, "then the Soviet Union will have reached its most important goal and you wouldn't really have reduced the possibility of a nuclear war in the world." He said "a severe inner political confrontation" would occur in West Germany, and there would be a "jolt" to the North Atlantic alliance that "could even increase the possibility of a future nuclear war."

This brought some booing from the convention floor. But when the show of hands came on the moratorium--offered as an amendment to a general resolution on security policy--the proposal was rejected by a clear margin.

The link with U.S. developments was accentuated by Oscar Lafontaine, the mayor of Saarbrueche and a leader of the moratorium drive. He told delegates a man named Norman Birnbaum, whom he identified as an adviser to Sen. Edward Kennedy (D-Mass.), "told me here that Kennedy's freeze campaign goes for Europe, too."

But Birnbaum, a Georgetown University professor, told a reporter he had been misinterpreted by Lafontaine. The U.S. connection was also strongly disputed by Peter Courterier, a West German minister of state, who told the convention that Lafontaine had "completely misrepresented" the position of Kennedy and Sen. Mark Hatfield (R-Ore.). He said their proposal was for a mutual, verifiable freeze on U.S. and Soviet arsenals and therefore quite different from the unilateral move being proposed by the Social Democratic left wing here.

In an important qualification that could bring problems for the North Atlantic Treaty Organization later, the party's final resolution opposed the notion that deployment of new Pershing II and cruise missiles in West Germany would be "automatic" if the U.S.-Soviet negotiations in Geneva show no result by the end of 1983.

This position contrasted with Schmidt's definite assertion to Soviet President Leonid Brezhnev in Bonn last November that unless the talks produce results by late next year, the Soviets would have to reckon with stationing of the new missiles.

The idea behind the party's more ambiguous position appeared to be to allow for at least the following possibility: if the negotiations seem close to a breakthrough later next year, the Social Democrats could back a moratorium then to extend the deadline for missile deployment.

The resolution said a special party congress would be held in the fall of 1983 to make a final decision on whether to accept the planned deployment of the missiles.

Leading party members explained that the aim of waiting to affirm the deployment was intended to put pressure on the United States, as well as the Soviet Union, to conclude an agreement that would make the U.S. deployment unnecessary. U.S.-Soviet talks on reducing medium-range nuclear weapons in Europe have been deadlocked since they began in December.

The resolution did carry a demand for a moratorium on "new rocket systems with shorter ranges" than those included in the NATO plans. This was apparently included to block a feared move by the United States to place neutron weapons in Europe. The resolution contained calls to outlaw neutron and chemical weapons.

Schmidt won a second victory today when party delegates defeated a left-wing motion for a two-year freeze on the construction of new nuclear power plants, which party leaders had worried would further set back the country's already politically stifled nuclear power industry.