Deputy Secretary of Defense Frank Carlucci touched the sensitive nerve of U.S.-West German relations when he remarked, at a closed-door session of the Armed States must never allow West German Chancellor Helmut Schmidt's left wing to "dominate" NATO's policy toward the Soviet Union.

Carlucci's comment came as Secretary of State Alexander Haig was struggling in Rome to keep Schmidt fixed to U.S. insistence on deployment of modern, medium-range nuclear missiles in Europe during arms-control talks with Moscow. Despite Haig's best efforts, West German plitical sensitivities forced deletion from the official communique of a flat warning that Soviet intervention in Poland would kill negotiations.

At issue here is what one Western European diplomat described to us as "the gravest threat" to the Western alliance: a Washington-Bonn split. Reaganite hardliners in the White House and Pentagon define it differently: Is the United States permitting Schmidt, out of deference to his precarious political position, too much elbow room against President Reagan's defense buildup in Europe and his challenge to Soviet aggression?

The tentative answer approved by Reagan is "no," representing a victory for Haig's emphasis on strengthening the Western alliance. But Carlucci's position at the May 4 Pentagon meeting reflects simmering discontent within the national security apparatus that will erupt if Schmidt fudges on his nuclear weapons commitments made at Rome.

Carlucci's remark was personally endorsed by Secretary of Defense Caspar Weinberger at the Pentagon meeting. Behind this were weeks of struggle inside the national security bureaucracy with hard-liners fighting a rear-guard action against negotiations. When agreement could not be reached at the working level, the problem was kicked upstairs to senior officials.

The struggle was continued there and finally resolved in a defeat for the hardliners, which was sealed at the April 30 National Security Council meeting. Haig was given instructions to proceed with the nuclear arms talks (though senior Pentagon civilians remained skeptical).

The president himself backed Haig at the NSC meeting. Just before Haig left for Rome, Reagan talked alone with him. Those talks confirmed Haig's strategy: Schmidt had to be kept on the U.S. track and nothing less than talks with Moscow could do that.

Reaganite hard-liners never have forgiven Schmidt's presumption in barging into a meeting with president-elect Reagan in Blair House last November, but their present concerns with the German chancellor are more substantial. They fear Schmidt, in order to protect himself from the activist left wing of his Social Democratic Party, will hedge on commitments to install the new medium-range nuclear weapons in his country. The hard-liners want a much stronger declaration of Reagan's aim to rearm Europe, in hope of shocking Schmidt into reducing the impact of the SPD left wing on Western defense strategy.

Schmidt's domestic problems are mounting. On May 3, regional SPD conference in the state of Baden-Wurttemburg voted overwhelmingly to force the party to review its 1979 approval of new nuclear weapons for NATO. It demanded an "immediate start" of arms talks with the Russians, vaguely set by the Rome NATO meeting for late this year.

That promises more political trouble for Schmidt. Some diplomats in Washington think he could not get a new SPD national conference to reaffirm the 1979 nuclear green light.

Polls commissioned by the U.S. International Communications Agency show opposition to nuclear arms rising to alarming peaks throughout Europe. It is particularly high in West Germany, with the latest poll showing 60 percent against deployment of modern nuclear weapons. The poll questions did not point out the huge Soviet superiority in European-based nuclear weapons, but the answers nevertheless disturb U.S. officials.

Also ominous for Schmidt are the West Berlin elections next week. The Christian Democratic Union is close to gaining control of the legislature after decades of SPD rule, possibly bringing in a CDU mayor for the first time since the early days of the Federal Republic. This rebuff to Schmidt's party could strengthen the influence of its well-organized left wing.

That is precisely what worries Reagan's hard-liners. Even though the president came down on Haig's side April 30, the door is closed on U.S. policy. If the chancellor fails fully to abide by the terms of the Rome agreement, the U.S. officials who say internal German partly politics are undermining Reagan foreign policy will have powerful ammunition against Haig.