A seething dispute has grown up within the Reagan administration and between the administration and hard-liners in Congress over arms control policy. And one question has become whether Eugene V. Rostow will continue as director of the U.S. Arms Control and Disarmament Agency (ACDA).

The immediate issue is Senate confirmation of two acting Rostow aides whose nominations have been held up for six months by Sen. Jesse Helms (R-N.C.) and others. Among other things, it is no longer clear how much political capital the White House is willing to expend in behalf of the nominees, whom Rostow has stoutly supported.

But the two nominations are only pawns in a broader battle over how the United States should proceed in the two sets of nuclear arms control talks under way with the Soviet Union in Geneva.

The point man in the conservative opposition is Helms, who has used confirmation fights in the past to win policy changes, particularly in foreign affairs. But within the administration, presidential national security affairs adviser William P. Clark also has become involved.

Late last month Clark, who is also regarded as a hard-liner on U.S.-Soviet relations, reportedly wrote a memo to Secretary of State George P. Shultz which some sources say was critical of Rostow and Paul Nitze, one of the U.S. negotiators in Geneva. The text has not surfaced.

Its existence was disclosed by Rostow opponents who portrayed it as a reprimand to Rostow from the president. Among other things the memo is said to have complained that Rostow and Nitze exceeded their instructions in dealing with the Soviet Union, which supporters of both men vehemently deny.

Members of the anti-Rostow forces do not speak with one voice. But Helms, the most active, says the Soviets have repeatedly violated arms control agreements in the past. He and others want the administration to lay out and put stress on these violations. Their great fear, they say, is an accord with the Soviets that lacks adequate provision for "verification."

On the opposite side, Rostow's supporters in the administ ration say the "thunder offstage" is really an attack "on the president himself" and "any possibility of realistic arms control" with the Soviet Union. What the arms control critics seek, Rostow supporters say, is some thing close to an official U.S. declaration that the Soviet Union reserves "the right to commit any crime, to lie, to cheat," as President Reagan once put ut.

And such an indictment, the Rostow side says, would destroy in advance the justification for any agreement with the Kremlin.

In this case, the dispute began at the outset of the administration, over the personnel and policy of the arms control agency. The agency is anything but a dovecote; its most senior officials were all leaders in the attack on the SALT II nuclear strategic arms limitation pact negotiated by the Carter administration.

Rostow, a conservative Democrat, has been reported to be on the verge of resignation over White House hesitancy to force a Senate vote on the two aides. However, a spokesman said last week Rostow "has no intention of resigning."

At stake are the jobs of Robert Grey, 46, acting but unconfirmed as deputy director, one of Rostow's closest friends and his executive assistant when Rostow was undersecretary of state in the Johnson administration, and of Norman Terrell, 49, the acting assistant director of the agency's bureau of nuclear and weap ons control.

Longtime professionals in government service, they have been assailed nevertheless as "Carter holdovers," out of tune ideologically with the 1980 Republican platform charge that a "coverup" of Soviet violations of arms control "began under the Nixon administration" and continued through the Ford and Carter administrations.

The consertatives' main mark against Grey is that his Foreign Service experience was interrupted in 1969-71 when he served as administrative assistant to Sen. Alan cranston (D-Calif.). Terrell, also a former member of the Foreign Service, has been under more personalized attack, and is the more vulnerable of the two politically.

At a meeting in the office of Senate Republican leader Howard H. Baker Jr. (Tenn.) in July, sources on both sides agree, Helms and fellow critics offered to confirm Grey if the administration would find Terrell another job elsewhere. Rostow instead strongly defended both subordinates.

Opponents say they have about 20 committed votes against Grey and Terrell. The administration could override those votes if it were prepared to pay the price. The threat, as usual in such cases, is a filibuster against confirmation. But in this instance, the threat exceeds normal dimensions. Helms, for example, told Rostow in a letter on July 29 after negotiations broke down:

"If these nominations are brought to the floor, I fear that it will trigger a lengthy and divisive debate on the nature of arms control, the history of SALT I and SALT II strateqic arms limitation talks completed in 1972 and 1979, the verifiability of those agreements, their legal standing, the Soviet record on Violations and the basic assumptions of START [President Reagan's proposals for arms reductions]."

Helms said such a debate could show the Soviet Union "there is little unanimity in the United States on START, and that any treaty so negotiated might meet the fate of SALT II," which failed to get ratified.

As one Rostow associate last week summed up the clash, "What was originally an admonitory effort by a few senators" to put pressure on the agency, "has turned into a jihad a holy war."

The central issue now is where the White House will come down.

The president in June did ask Baker to act on the Grey-Terrell appointments, sources in the Rostow camp and the Senate leadership agree, although Grey-Terrell opponents have denied that.

Nevertheless, according to several White House officials last week, the administration is not prepared to spend further political capital on this issue. The president in recent days has been clearly seeking to repair strained relations with his right-wing supporters, and notably with Helms and other opponents of Rostow.

In addition, Rostow's style is being criticized by some White House officials as too inflexible and verbose for the taste of the president and his senior advisers.

And then there is the Clark memorandum.

Critics charactarize this memorandum as "a reprimand" of Rostow, for exceeding his instructions in dealing with the Soviet Union, and for failing in some instances to obtain adequate clearance of his statements and actions with other departments and agencies.

Nitze, who is 75 and is the most experienced American arms control negotiator, heads the American delegation in Geneva, in negotiations with the Soviet Union on limiting medium-range nuclear weapons based in Europe.

Nitze, according to other sources, expressed indignation that he was accused of exceeding his instructions in talks in Geneva. Nitze reportedly countered that the accusation came from novices evidently unaware that it is standard negotiating practice to ask hypothetical questions to draw out Soviet bargaining positions. One administration source said Nitze resolved the questions raised about him.

Rostow spoke in Los Angeles last week about the two sets of nuclear negotiations in Geneva, one conducted by Nitze and the other by retired Gen. Edward L. Rowny. In these negotiations, Rostow stressed, "the United States has made it clear that verification measures capable of assuring compliance are indispensable."