A split has developed in the administration over whether President Reagan should move away from his "zero option" proposal for reducing European-based nuclear arms, which calls for the Soviet Union to dismantle all its medium-range missiles and the United States to cancel deployment of the new Pershing II and ground-launched cruise missiles.

On one side in inter-agency working group deliberations about how to advise Reagan, according to informed administration sources, are those who believe there is an opportunity beginning in the next round of negotiations with the Soviet Union in Geneva to achieve agreement on a substantial--but not total -- reduction of Soviet SS20 nuclear missiles aimed at western Europe.

Those holding this view, who are said to include some officials close to the negotiations, reportedly believe the Soviets are prepared to destroy some of their SS20s and older missiles while moving other SS20s to eastern Russia out of range of targets in NATO countries.

Such an agreement, however, the sources said, would leave some missiles remaining on both sides in Europe. This would require Reagan to abandon his "zero option" insistence on the Soviets agreeing to dismantle all their SS20 and other medium-range missiles.

Arguing against this are administration officials, mostly in the Pentagon, who believe the security of NATO countries cannot be assured without elimination of the 324 SS20s so far deployed by the Soviets along with 300 older SS4 and SS5 missiles.

They contend that the almost 1,000 warheads on these missiles constitute much more than the Soviets need for their security, so that any reductions short of nothing would still leave them with a nuclear missile advantage over NATO in western Europe.

"I haven't seen any significant change by the Soviets other than that which would leave them with missiles and us with zero," one Pentagon official holding this view said of the latest round of talks in Geneva, which ended Nov. 29. "Maybe some people see straws in the wind," he added. "But I don't think the Russians will change until we come closer to deployment [of Pershing II and cruise missiles]."

Reagan ultimately may have to settle this conflict before the administration drafts new instructions for its negotiator, Paul H. Nitze, to take back to the next round of talks scheduled to begin Jan. 27 in Geneva.

The negotiations were prompted by a NATO decision to deploy 108 American Pershing II and 464 ground-launched cruise missiles in western Europe beginning in December, 1983. The Soviets have opposed this deployment of missiles that would be able to hit Russian targets within minutes of launch.

The United States has sought to limit the negotiations to missiles, while the Soviets want them to include U.S. bombers and British and French nuclear weapons. The Soviets also want to omit SS20 missiles aimed at targets in China from bases in the eastern Soviet Union.

The talks that ended Nov. 29 were described by sources earlier this week as productive, with the Soviets making important adjustments in their earlier positions. Some participants "believe there is an opportunity now for a breakthrough," one official said yesterday.

However, Nitze reportedly was stopped last October by Washington from pursuing, during informal meetings with the Soviets, their idea of moving SS20s east of the Ural Mountains. Strong backers in the administration for the zero option proposal reportedly objected that Nitze had exceeded his instructions, according to informed sources, and the matter was dropped after White House national security affairs adviser William P. Clark sent an inquiry about it to Nitze in Geneva.

Some officials believe the Soviet Union may generate new pressure on the United States to change its position by announcing before the next round of talks that it will include in a new negotiating position reductions not only in the Soviet missile force, but also in its nuclear-capable aircraft and battlefield weapons.

This move could be aimed at West German voters going to the polls in March. The Bonn government of Chancellor Helmut Kohl supports the zero option position but wants the Reagan administration to be prepared to alter it if an opening develops in the negotiations. His socialist opposition has said deployment of the NATO missiles should be delayed as long as negotiations are under way.

Although NATO foreign ministers meeting in Brussels yesterday reaffirmed their support for the zero option, NATO Secretary-General Joseph Luns said at a NATO defense minister' meeting earlier this month that "the United States never said it was zero option or nothing."

Earlier, West German Defense Minister Manfred Woerner was quoted making a similar statement. Both he and Luns had been briefed on the latest round of Geneva negotiations before making their statements.