U.S. and Soviet negotiators return to the nuclear bargaining table in Geneva on Tuesday in deep disagreement not just about the numbers and destructive power of weapons but about the underlying ground rules of the competition the two superpowers have carried on since the dawn of the nuclear age.

The drive to curb that competition through negotiations, which commenced 15 years ago last November, has produced several treaties and lesser accords that may have regulated the growth of offensive nuclear arsenals but have failed to stop or reverse it.

Now, for the first time since 1972, the two sides also are far apart on the theory as well as the reality of the competition. The defensive aspect of the nuclear arms race, which has been limited since ratification of the 1972 Antiballistic Missile (ABM) Treaty, has become the cockpit of conflict due to President Reagan's Strategic Defense Initiative (SDI) or "Star Wars" plan.

The U.S. "strategic concept" for the future military relationship of the superpowers, to be presented for the second time to the Soviets in the negotiations, envisions a world in which "nonnuclear defenses against strategic offensive arms" play an ever-growing role.

Over the next 10 years, according to the U.S. concept, Washington and Moscow should aim for "radical reductions" in missile forces to facilitate the transition to the new era.

A major aim of the U.S. negotiators at Geneva is to persuade the Soviets to accept at least the theory of such a "defense-dominant" world of the 21st century. But the highest priority of the Soviet negotiators will be to thwart Star Wars development by shoring up restrictions on testing of antimissile weapons and negotiating new ones.

The new U.S.-Soviet negotiations Tuesday in Geneva are to range over three broad areas: space and defense arms, long-range nuclear offensive arms and intermediate-range nuclear forces in Europe.

Soviet Foreign Minister Andrei A. Gromyko, who established the guidelines for the new talks in Jan. 7-8 meetings with Secretary of State George P. Shultz, said last month that "any reduction, to say nothing of elimination of nuclear weapons, would be out of the question" while the United States goes ahead in space.

Soviet strategy in establishing the guidelines for the talks -- and by all indications, in the negotiations to come -- is to link cutbacks in offensive arms with cutbacks in the U.S. missile defense program. The Negotiators' Objectives

In the Soviet view, Reagan's Star Wars plan is "aggressive" rather than "defensive" in that it would alter radically the strategic balance in favor of the United States by neutralizing Moscow's superiority in offensive nuclear missiles.

U.S. negotiators, however, left Washington on Friday with an array of ambitious objectives for the new talks:

* Major reductions in long-range offensive nuclear arms, which have been growing steadily in number, power, sophistication and accuracy since the dawn of the atomic age in 1945, despite all efforts to curb them.

* Negotiated cutbacks in the current and planned deployments by the two sides of intermediate-range nuclear missiles in Europe.

* Reversal of "the erosion" of the 1972 ABM treaty, which is attributed by the Reagan administration to Soviet violations of the letter and spirit of that pact and the failure of the two sides to make hoped-for cuts in offensive weapons.

* Acceptance by the Soviets of Reagan's Star Wars program at least as a research effort.

Each of these aims is imposing enough to consume years of tough bargaining. Taken together, they could easily consume the remainder of Reagan's second four years in office, in the absence of a dramatic change of policy in the Kremlin or Reagan's willingness to bargain away his plans for space-based missile defense.

The new Geneva negotiations will be hostage to an imposing series of uncertainties that can deeply affect their outcome.

Konstantin U. Chernenko, 73, the Soviet leader, is in poor health and may have only a few more months to live. Gromyko, who has been directing arms negotiation policy with growing authority in the past several years, is 75. The leadership of the Soviet military, which plays a key role, is in flux.

Reagan's real intentions toward the Star Wars program are under debate in the administration. The plan, unveiled with little preparation in March 1983, in the past several months has become the apple of the president's eye and allegiance to it a test of loyalty among foreign and defense-policy officials.

A question being asked increasingly in private is whether, despite his rhetoric, Reagan would accept negotiated limits that tend to compromise the plan if the Soviets offer major offensive weapons reductions in return.

Perhaps the greatest uncertainty of all is the validity of Star Wars itself. It is a long-term research program of massive dimensions, dwarfing in scale and difficulty the Manhattan project that produced the atomic bomb and the Apollo project that landed men on the moon and returned them safely. Seven to 10 years may pass before even the concepts involved in Star Wars can be tested fully and a decision made about their ability to work in weapons.

Rarely has such an untested idea of such questionable military application so swiftly overwhelmed prevailing political thinking and negotiating strategies in the nuclear weapons field. One of the few precedents is the idea of a disarming "first strike" by Soviet warheads against the U.S. land-based missiles arrayed in the "vulnerable" silos.

The Reagan administration grappled with the "vulnerability" question without success in its first term. Having failed to find a workable solution, it now looks to the Star Wars plan, to be validated only in the decades to come.

Reagan's vision of a world in which nuclear missiles are rendered impotent had immediate impact in the Kremlin, where Yuri V. Andropov, then the Soviet leader, denounced the plan within four days of its March 1983 announcement. Andropov called it "a bid to disarm the Soviet Union in the face of a U.S. nuclear threat," a Soviet viewpoint that has not changed in the two years since. Reconciling U.S. Policy ---

Most military strategists and strategic theorists in the United States, including many in the administration itself, did not take Reagan's proposal seriously until late last year. With Reagan reelected by a landslide and signs of an approaching renewal of U.S.-Soviet arms negotiations, the SDI suddenly became as prominent in Washington as it had been in Moscow.

It soon became clear that a way had to be found to reconcile Reagan's initial arms control policy -- deep cuts in Soviet offensive missiles through negotiations -- with the new aim of building a U.S. antimissile shield to neutralize the Soviet nuclear threat. On the face of it, the Soviets were being asked to accept two things at once to their deep disadvantage. The answer came from Paul H. Nitze, a veteran of four decades of strategic policy-making and author in 1950 of NSC 68, the first long-term U.S. strategy for dealing with the Soviet Union. Last December, Nitze, now a senior U.S. adviser on arms negotiations, drew up the "strategic concept" that seeks to bring the two arms control ideas together.

With assistance from National Security Council staff members, Nitze fit the two concepts together on the basis of phasing. He drafted a statement of U.S. strategy that has become the basis for the administration's new approach to arms control. Boiled down to four sentences, it was presented by Shultz to Gromyko in Geneva two months ago and subsequently was made public in U.S. speeches and official documents. No Nuclear Arms in 'Ultimate Period'

The concept was that during the next 10 years (Nitze originally wrote five to 10, but was persuaded to stretch the period) the United States would seek "a radical reduction in the power of existing and planned offensive nuclear arms," at the same time seeking "stabilization of the relationship between offensive and defensive nuclear arms."

The following transition phase would see nuclear offensive weapons reduced and defensive antimissile weapons coming into their own.

Finally, in what Nitze calls "the ultimate period," the withering of the military threats "could lead to the eventual elimination of all nuclear arms, both offensive and defensive."

The Soviets have made clear that they do not buy the "strategic concept," and that they have little zest for the "seminar" on the offense-defense problem and the long-term future that has been promised them at Geneva.

Col. Gen. Nicholai Chervov, chief of the Soviet general staff's section on arms control negotiations, said here last week that, "We're not going to wait for such a seminar. You are working on your system and that means we'll start perfecting our strategic offensive arms."

But the more the Soviets protest, the more administration leaders feel confident, at times even cocky, that the introduction of Star Wars has put Moscow on the spot and greatly enhanced U.S. bargaining power.

Many officials believe that the wish to thwart SDI was a powerful incentive for the Soviets to return to negotiations. They noted that during past 15 months, Moscow continued to push for negotiations about space weapons even while refusing to bargain about strategic and intermediate-range offensive weapons.

In the inner councils of the administration, disagreement persists about how to make use of this perceived bargaining power once the talks are under way. The unanswered question is whether, and to what extent, Reagan is willing to trade part of his vision for some of the reductions he seeks in the existing era.

Disagreements in the administration during Reagan's first term were one of several factors that have been blamed widely for lack of any accords or even substantial progress toward them in arms control. Recently administration policy-making on arms negotiations has been raised to the top level with the president participating more fully. The differences have tended to lessen or be submerged, but they may reappear if the Soviets show readiness at Geneva to bargain in earnest.

Arms control through negotiations, which captured the world's imagination during the 1970s, ran into increasing trouble in the first half of the 1980s. Increasing development and deployment of new weapons far outran the tenuous limits that negotiators sought to impose at the bargaining table.

Some of these weapons are more difficult to observe and count through spy satellites. The administration, meanwhile, increasingly has accused that the Soviets have violated pacts that are in force.

Tens of thousands of nuclear weapons now deployed or being produced by the two sides are not covered by the Geneva negotiations, including sea-launched cruise missiles and shorter-range battlefield weapons.

For all the difficulties and shortcomings of arms control, Washington and Moscow have found it convenient and possibly necessary to return to negotiations.

Some of the reasons behind the arms control drive are political. The Reagan administration, like others before it, is using the negotiations to argue against congressional cuts in major nuclear weapons programs.

The Soviets find the talks important to their political position, especially in Western Europe, where peace movements took a beating after the Soviets walked out of earlier negotiations. The Soviets also seem to find the negotiations useful in easing their Eastern European allies' concerns about increasing deployment of Soviet weapons on their soil. A Bid for Confidential Bargaining

In the last round of arms negotiations, in 1981 to 1983, the two sides publicized their bargaining positions in a way that suggested that the real audience was public opinion. This time the Reagan administration is beginning by withholding details of its initial positions in a bid for confidential bargaining. But officials are reported to be preparing a publicity blitz in case the Soviets go public.

Arriving in Geneva early yesterday, U.S. chief negotiator Max M. Kampelman promised to "thoroughly and responsibly explore all avenues" that could lead to a successful result.

"The differences between the United States and the Soviet Union are profound," Kampelman said. "It would be folly to expect them to be bridged overnight."

The consensus of officials dealing with arms control in Washington, as well of nongovernmental experts, is that the negotiations to be launched Tuesday will take a long time to produce positive results.