The true threat to Senate ratification of an intermediate-range nuclear treaty is not the shift away from on-site inspection but Senate anger that Moscow continues to violate existing treaties, including a potentially grave new ABM treaty violation now under White House study.

The prospect that an INF treaty might collide with the 34 Senate votes needed to defeat it next year has been the subject of several private warnings from conservative Republicans. The most recent came in a confidential letter to President Reagan last month from Sen. Jesse Helms, senior Foreign Relations Committee Republican.

Helms' classified letter, according to Senate colleagues, squarely raised this issue: without Soviet agreement to end current violations of existing treaties, ratification of any new treaty cannot be counted on.

Reagan mentioned the compliance issue, but in unusually low-key terms, in his Los Angeles speech Wednesday. He said only that ''we need to seek compliance with existing agreements all too often violated by the U.S.S.R.'' He made no mention of the new issue that went to his desk only this week.

This mild treatment of the compliance issue, which was elevated by the Reagan administration to unprecedented prominence starting in 1984, was uncharacteristic. That may reflect the president's intentions not to let any barrier build up that might block an INF treaty from being signed in the next few months.

Only last March 10, the president was far more forceful. In his obligatory annual report to Congress on the compliance issue, Reagan said, ''we have deep, continuing concerns about Soviet noncompliance with the ABM {anti-ballistic missile} treaty.'' Referring to what is probably the single worst Soviet violation, the giant radar now being completed not near a Soviet border, as required under the treaty, but near the Siberian city of Krasnoyarsk, Reagan said, ''the Soviets have not corrected their outstanding violation.''

The administration's decision to relax its earlier demands for extensive on-site inspection to ensure compliance with an INF treaty demonstrates nothing in terms of expectations of future Soviet performance. It simply means the on-site inspection system devised by arms control director Kenneth Adelman lost its relevance when Moscow agreed to total elimination of all its SS-20 intermediate-range missiles.

If 100 warheads had been allowed to remain on each side, as the Soviets originally insisted, the Arms Control and Disarmament Agency's inspection system would have been essential. Inspectors would have checked production lines for spare parts or even new replacement missiles. They would have watched the warhead population in missile housing centers, called ''garages.''

But that task eased considerably with the decision to destroy all Soviet SS-20 missiles and all American Pershing II and ground-launched intermediate-range missiles. The most ominous question facing the United States is no longer production lines and ''garages'' but exactly how many SS-20 missiles the Soviets actually have. The Soviets claim to have 441, but that figure is not accepted by many top officials here.

''There are some bits and pieces of uncertainty about that number,'' one administration official told us. The uncertainty arises partly from the fact that the United States does not know what happened to at least 27 SS-20s taken out of European Russia last year. The ''garages'' were dismantled and the missiles removed, but U.S. intelligence does not know where they are.

There is no known way, however, to find these ''lost'' missiles by on-site inspection, or to verify beyond question that they are included in the 441 to be destroyed under U.S. inspection. If they are in a Siberian warehouse, there is no way to find them. A million on-site inspectors would not locate a hiding place.

This lack of confidence in the Soviet pledge to destroy all its SS-20s under the forthcoming INF treaty, whether 441 or many more, is the direct result of Soviet cheating on existing treaties. That is why suspicious senators -- and some equally suspicious administration officials -- want Moscow to make a dramatic overt act, such as stopping work on the Krasnoyarsk radar, to show a change of heart on the compliance issue. Although highly unlikely, such an act is not ruled out.

The new violation now under White House study seems to involve a somewhat ambiguous radar development. It appears to be one more piece of evidence pointing to the much-feared possibility of an early Soviet breakout from the flat ABM prohibition against a nationwide antimissile defense system.

But even if it proves to be something less, the Kremlin's failure to correct one single violation charged against it by Ronald Reagan makes ratification of any future treaty a questionable endeavor.