The United States and the Soviet Union failed to settle their differences on a new treaty limiting their strategic nuclear arzenals yesterday, but scheduled another round of high-level talks in Moscow in the drive to close the gap.

After meeting for nearly three hours with Soviet Foreign Minister Andrei A. Gromyko in an unusual Sunday morning session, Secretary of State Cyrus R. Vance announced he will go to Moscow in the latter half of October to continue the discussions.

"Obviously we have not reached agreement on all the issues," Vance said.

The secretary said "many new ideas" were proposed in the talks with Gromyko during the past week and repeated that there has been "constructive movement" toward full agreement. He continued to predict that the new strategic arms limitation talks (SALT) treaty will be concluded before the end of this year.

Vance reported that the subject of a summit meeting between President Carter and Soviet President Leonid Brezhnev has been discussed, but would give no details. A meeting of the two leaders is expected as the concluding stage or signing ceremony of a Salt II treaty.

The inconclusive end of the present round of negotiations did not dampen optimism of administration officials that final bargaining is at hand in the six-year search for a binding, detailed treaty on the limitation of offensive strategic armaments. Several officials expressed the view that the Vance trip to Moscow is likely to produce a final agreement in principle on a treaty and a summit meeting for its signing.

Differences between the two nations on fligt testing and deployment of "new types" of intercontinental ballistic missiles, one of the most important outstanding issues, were reported by informed sources to have been virtually settled in last week's negotiations.

The most significant remaining questions, these sources said, involve details of the proposed limitations on U.S. developed cruise missiles and Soviet Backfire bombers. Broad disagreements on the same two issues stymied a U.S.-Soviet agreement on strategic arms during the two years of the Ford administration.

At this stage, Gromyko has said repeatedly and Vance confirmed yesterday, the proposals of the two sides on the remaining issues are interlocked in intricate bargaining. According to officials, this involves proposed trade-offs of concessions and also agreements on some issues contingent on the successful resolution of other issues.

In such circumstances, final agreement on the remaining issues could fall into place quickly if political leaders decide to compromise in a spirit of give and take. On the other hand, an unexpected deadlock on a few remaining issues could undercut the structure of tentatively agreed provisions and collapse them like a house of cards, officials warned.

Vance and other members of the U.S. negotiating team had been prepared to continue the talks with Gromyko through the day yesterday and part of today if such sessions held promise of a breakthrough toward final agreement. The decision to end the talks at midday yesterday and schedule another round later was ascribed by informed sources to Gromyko's inability to obtain positive answers from Moscow overnight to a new set of U.S. proposals outlined Saturday at the White House by Carter.

Accompanying Gromyko to the State Department session was Ambassador to the United States Anatoliy Dobrynin, Deputy Minister of Foreign Affairs G. M. Korniyenko and interpreter Victor Sukhodrev.

In addition to Vance, the U.S. participants were Director Paul C. Warnke of the Arms Control and Disarmament Agency, Ambassador to the Soviet Union Malcolm Toon and interpreter William Krimer.

Following the meeting with Gromyko, Vance briefed representatives of the secretary of defense, Joint Chiefs of Staff and National Security Council, according to the State Department.

The agreements under negotiation are: a SALT II treaty limiting offensive nuclear arms to run until Dec. 31, 1985; a three-year protocol covering several of the most sensitive issues; and a statement of principles for negotiating a SALT III treaty aimed at deeper cuts in the arsenals of the two superpowers.