THE RULES of the game of Soviet-American relations seemed to be changing last week, as an aggressive, charming and supremely confident Soviet leader visited Washington. It was hard, watching Mikhail Gorbachev, not to be impressed by the force of his personality, and his visit led many Americans to hope that a more peaceful era lies ahead.

But the transition to this new era of superpower relations could be unstable and dangerous, as Moscow and Washington grope to understand just what the new rules are and how they apply to the real-world political and military problems that both nations face. Events have been moving so fast in recent months that American officials have barely had time to assess Gorbachev's new policies, let alone respond sensibly to them.

Under the old rules of the game, a kind of stability had evolved. The two superpowers maintained vast nuclear arsenals that made full-scale war irrational; they competed ideologically and exchanged shrill rhetoric, but the underlying relationship was remarkably stable and predictable; they inhabited an essentially bipolar world, and both kept their economically dependent clients more or less in line. Even when one of the superpowers went to war, as in Vietnam or Afghanistan, the rules of the game seemed able to contain the conflict within acceptable limits.

The system proved to be remarkably resilient, despite public concern that it contained the seeds of nuclear holocaust. In the 1970s, thriller writers speculated that the sinking of a supertanker in the Persian Gulf could set of a global collapse and World War III, and readers found such scenarios perfectly plausible. By 1987, a supertanker was being attacked in the Gulf almost every week and no one blinked an eye.

The new Soviet-American game that is emerging will look different, with different rules. The biggest change, thus far, is the impact of Gorbachev himself. Anyone watching television last week understood that Gorbachev is a formidable politician: at once seductive, overbearing, humorous, and bombastic. Clearly, this is an immensely clever and ambitious man, someone who can play the lyre of politics on all its strings. But it's also clear that Gorbachev himself is far more modern and self-assured than the country he leads, and this disjunction could cause problems.

For American national-security planners, who will be around long after Ronald Reagan is gone, and perhaps after Gorbachev is gone as well, the changing rules of the game -- in diplomacy and arms-control -- hold great promise, but also potential pitfalls. Here are some of the issues that may bear further study after Gorbachev's remarkable week in Washington:

In diplomacy, the superpowers seem to be reversing roles. Once it was the Americans who led the way, with new arms-control proposals and aggressive diplomacy in the Middle East and other hotspots. The Soviets played the stodgy role, disdaining the shuttle diplomacy of a Henry Kissinger and waiting for the other side to make concessions. Gorbachev has changed that picture. Now it's the Soviets who are the aggressive diplomats, offering concession after concession in their quest for arms-control agreements. This Soviet approach offers obvious short-term benefits for the United States. But the role reversal poses some long-run problems. With his enthusiastic diplomacy, Gorbachev is forcing the United States into a reactive mode. He and his diplomats are all over the map, while America watches and waits to see what he'll do next.

Would this role reversal prove stable in a crisis? Would the Soviets, for example, give way in a new Mideast war as they did in 1973.

The old bipolar world, dominated by the superpowers, is giving way to a multipolar world, in which other nations -- such as China, Japan and South Korea, to name just a few -- will play a larger role. The economic growth of these other nations comes at a time when the United States and the Soviet Union both have serious economic problems. The late Herman Kahn foresaw these developments in a 1983 paper on "Multipolarity and Stability." He predicted that by the year 2000, there would be seven nations with trillion-dollar economies -- the United States, Japan, the Soviet Union, China, Germany, France and Brazil -- all capable of projecting political and economic power beyond their borders, five of them possessing nuclear weapons. Kahn argued that this multipolar world would eventually be safer than a bipolar one, but he worried about the transition from two superpowers to seven. Along the way, amid shifting alliances and coalitions, war could erupt.

Gorbachev could seek to exploit this trend toward multipolarity by warming relations with China or Japan -- or perhaps both. A full Soviet-Chinese rapprochment is probably unlikely, but if it happened it would erode the most important American diplomatic achievement of the 1970s.

In arms control, the rules of the game are changing radically. The superpowers have already agreed in principal to reduce their intercontinental-range, nuclear-armed ballistic missiles and bombers by 50 percent. Such an agreement, if it can be pinned down at next year's summit in Moscow, will undoubtedly meet with popular acclaim. It could reduce the fear that most drives the arms race; that one side will have the ability to launch a successful surprise attack against the other.

But such reductions also could add to uncertainty in a future crisis -- putting nuclear arsenals on more of a hair-trigger -- if the weapons each side has left are not carefully chosen. America's remaining land-based missiles and missile-firing submarines could become more vulnerable to Soviet attack than they are today if we are not careful.

Similarly, the just-completed INF treaty eliminating both sides' intermediate-range missiles from Europe and Asia is widely welcomed. But it raises critical questions about how the NATO alliance will cope with the larger conventional forces of the Soviet Union and its Warsaw Pact allies. And, what will be the long-term effect on West Germany, the military linchpin of the European alliance, and how will a future Bonn government behave in a crisis?

America's missile-carrying submarines -- probably our most important strategic asset -- could be compromised by the wrong START agreement. Although there is not yet agreement on how many weapons of each type would be allowed under a 50 percent cutback, it seems clear that the United States would be left, at best, with about 18 of the Trident missile-firing submarines. Each vessel carries 24 missiles with eight warheads on each missile, and most analysts suspect we might be allowed roughly 3,600 such warheads in a new agreement.

These vessels are the heart of the American deterrent force because they are hard for an enemy to find. The missiles, therefore, are secure and need not be fired quickly in a crisis. For more than a decade, the U.S. Navy had 41 such submarines of earlier models. Today we have 37. So, under START limits, that fleet would be cut in half. Furthermore, perhaps only ten or 12 of the future 18 would be at sea at any one time. As a result, the problem for the Soviets in trying to locate and keep track of these vessels would be greatly reduced. The Soviets have well over 100 attack submarines now, the kind that hunt down the missile-firing variety, and there are no limits on production of these attack subs. The Soviets, therefore, would be able to assign even greater numbers of attack submarines than is now the case to shadow, and attack in a showdown, ever smaller numbers of missile-firing submarines that are the core of the remaining U.S. nuclear deterrent.

America's land-based deterrent force may also be vulnerable in the new world of Soviet-American relations, in some measure because of past Reagan administration actions.

Ironically, Gorbachev's ability to make sweeping arms reduction proposals is due in part to the long and well-paced missile build-up over a dozen years by his unimaginative but steady predecessor, Leonid Brezhnev. The Reagan administration, on the other hand, scrapped for political reasons a Carter administration plan for 200 new MX land-based missiles based in a shell game arrangement that would have complicated any attacker's plans. The Reagan idea to junk that plan and keep the missiles, each of which carries ten warheads, in underground silos at fixed locations was so ill thought of by Congress that the number was cut to 50 missiles.

Using the arithmetic that may flow from a START agreement that mandates 50 percent cuts, the United States might be left with only 300-400 land-based missiles, MXs and older Minutemen. That might make it possible for the Soviets to target four or five warheads from their biggest and most accurate missile -- the SS-18 -- against each one of the remaining U.S. missile silos. This would be an increase over what they can do now. The more warheads aimed at a single target, the greater the confidence in knocking it out.

With both the Trident and the MX, the U.S. would be paying a price for putting a lot of its nuclear eggs, or warheads, in a small number of baskets.

The answer to making land-based missiles less vulnerable to attack, many arms specialists have argued, is to develop small mobile missiles that can't be easily targeted. Yet the U.S. position currently is to ban such weapons. True, the Air Force is pursuing development of a small mobile missile, but in a manner that has been lukewarm at best.

The INF agreement, as critics have repeatedly argued, will cause problems for NATO. Once the U.S. Pershing II and cruise missiles are withdrawn under the just signed INF treaty, NATO will have difficulty coping with the numerically-superior conventional forces of the Soviet Union and Warsaw Pact.

(A START agreement might also hinder U.S. ability to fight a conventional war in Europe and elsewhere. For example, if nuclear-capable B-52 bombers are eliminated in large numbers from a nuclear-arms agreement, those planes might not be available for carrying non-nuclear, high-explosive bombs in a conventional war, as was done in Vietnam.)

But a broader question is whether Gorbachev's real intention is to gradually reduce American military power -- and thus U.S. influence -- in western Europe. It is likely that the Soviet leader will soon advance new troop reduction proposals which will have great popular appeal within Europe. With so many troops and a relatively easy trip back to the Soviet Union, it will be easy for Gorbachev to make such offers. If they are accepted and U.S. forces come back across the ocean, the process of de-linking the United States from western Europe will continue.

Will our European allies permit us to use the weapons that will remain after INF? As matters now stand, NATO arsenals still contain a few thousand small nuclear weapons, such as artillery shells, as a deterrent to a Soviet conventional attack. In the event of an unforeseen crisis and attack, these battlefield weapons would go off on German soil, east and west. The Germans know this and many people there want these weapons out. NATO is resisting. But even if the weapons stay, Bonn might balk at using them during such a crisis as pressure built from a German public that knew that the nuclear threshold was on their doorstep alone?

These and other questions are difficult for American strategists to answer. But the questions will have to be answered in the remaining year of the Reagan administration -- or, to be precise, in the five or six months that are left before a Moscow summit.

David Ignatius is associate editor of the Washington Post and editor of Outlook. Michael Getler is the Post's Assistant Managing Editor for foreign news.