Three weeks of public maneuvering over Soviet combat troops in Cuba has led the Carter administration into a chancy diplomatic negotiation with the Russians, endangered Senate ratification of SALT II and threatens to inaugurate a new era of tension and confrontation between the nuclear superpowers.

The cause is a brigade of about 2,800 Russians, armed with rifles, field artillery and about 40 tanks. They and their predecessors may have been in Cuba as long as 17 years."Militarily it doesn't amount to anything," President Carter is reported to have told congressional leaders Thursday. He added, "Politically it is very important." News Analysis

The startling disparity between the size of the problem and the scope of the ability of the Carter administration to manage the nation's affairs.

Some of the administration's strongest supporters on Capitol Hill are saying, more in sorrow than in anger, that the affair has been blown out of proportion and has been badly mishandled.

Few could disagree with the president about the political importance of the situation. It has reopened the old struggle within the administration and the American body politic about the nature of the Russians and the ways of dealing with them, giving new ammunition to those who fear the worst.

And it has created openings for Carter's rivals for power as the weakened president heads into a fight for political survival.

How all this came about is still mystifying some of those who are most deeply involved in the attempt to find a resolution. Among the elements were: growing strength and self-assurance of the Russians; irregular and insufficient attention of U.S. intelligence; a climate of official indiscipline in which embarrassing secrets leak quickly to politicians and the press; lack of political savvy and strength at the top of the Carter administration; eagerness of office-seekers to exploit foreign policy troubles in an era when Congress has greater information and control than before and the unfortunate fact that the trouble developed while half of official Washington was on vacation.

It is of fundamental importance that the Soviet brigade was uncovered and announced in the midst of the SALT II debate. At another time, the same revelations might have generated a different set of responses.

Looking forward is no easier than looking back at what has happened. Nonetheless, statements and hints of informed officials lead to these conclusions:

First, there appears to be a less-than-ever chance that the Soviet leadership will accommodate the United States with a diplomatic solution. The higher the United States demand in public and private, the slimmer is the chance for a negotiated solution.

While the withdrawal from Cuba of the Soviet brigade remains the United States objective, the Soviet have said at least once in the diplomatic talks that this is out of the question.

The United States has now presented to Moscow, through Secretary of State Cyrus R. Vance's talks with Soviet Ambassador Anatoliy F. Dobrynin, suggestions for solutions short of complete withdrawal. These are believed to include dissolution of the Soviet brigade, reassignment of key personnel to advisory duty and distribution of its artillery and tanks to Cuban forces.

The upshot would be the end of the Soviet "combat capability" which U.S. officials have said is crucial, even if most of the Russian soldiers remain for the time being.

The Soviets appear to be alternatively baffled, indignant and suspicious about the U.S. demands, and they have disseminated for western consumption the view that they will not be intimidated in "crude" fashion.

They have much at stake. SALT II has been seen as the crowning achievement of Leonid Brezhnev's fading leadership, although SALT's value has been marked down to some degree by the U.S. arms hikes which the Senate has demanded to accompany the treaty, and by the lack of an aura of detente.

At the same time, Soviet relations with Fidel Castro, an increasingly valuable ally worldwide, are on the line in the resolution of the current problem. There is little doubt that the Russian troops are in Cuba under explicit arrangements with Castro.

Second, Carter has made it increasingly plain, as he told out-of-town editors in an interview released yesterday, that he will order "appropriate action" if the brigade issue cannot be resolved diplomatically.

White House officials have described these actions are "offsetting and compensatory" and said that they are not intended to be militarily threatening to the Soviet Union or to Cuba. There has been much discussion at high levels of these potential actions, but reportedly no final decisions.

One group of officials within the administration has counseled that the countermoves should be tailored to the regional problem of Cuba. This would suggest a beefing up of the U.S. naval presence in the Caribbean and of U.S. troops at Guantanamo Bay, and possibly a resumption of U.S. spy plane overflights of Cuba.

Another type of counteraction under study involves economic and political penalties, such as reduced sales of U.S. high technology to Russia, continued denial of most favored nation trade treatment and the like. For domestic reasons, wheat sales seem to have been ruled out as a lever.

More controversial is a set of countermeasures directed to areas of vital interest and great sensitivity to Moscow. These include U.S. sales of arms or other militarily important items to China, the Soviet neighbor and archrival. Such moves have their backers within administration. They would be likely to provoke a sharp and strong reaction from the Russians.

Finally, there is no doubt that the SALT II treaty and the U.S.-Soviet relations on a broad front will be deeply affected by the way the dispute is resolved.

A negotiated solution, if unambiguous, could enhance the chances of SALT ratification and strengthen Soviet-American relations, at least from this end. A more ambiguous solution, which is more likely, would be controversial politically and probably positive in overall impact.

More serious would be the failure of negotiations and the imposition of U.S. countermeasures. Several senior lawmakers have said publicly that there is no chance for SALT II unless the Soviets take action to resolve the problem of the brigade. The defeat or limbo of SALT II would in turn generate major consequences in Moscow.

Some high administration officials who advocate tough action against the Russians are dissenters from this line of thought. These officials argue that tough action would permit a politically strengthened Carter to tell the Senate that the Russian brigade has been neutralized, and to ask for the approval of SALT.

The problem is that U.S. compensatory actions could generate an even stronger anti-Soviet climate at home as even deeds from the Kremlin. An action and reaction cycle of this sort would certainly doom the SALT treaty, in the view of experienced observers of the Senate.

More than that, such a cycle of challenge and response can feed upon itself in a free-for-all U.S. election season which coincides with a period of leadership transition in the Soviet Union. At that point, the argument over 2,800 troops will have generated tensions and troubles far more serious than anyone foresaw at its outset three weeks ago.