The Soviet Union and the United States rejected each other's proposals for a new accord on curbing strategic nuclear weapons tonight, ending their Kremlin talks in gloom and plunging American-Soviet detente to a new level of uncertainty.

This poses the first grave global diplomatic test for the Carter administration. It may have profound long-term consequences for the Soviet Union as well. Each nation has repeatedly cited nuclear arms control as the fundamental issue in the East-West relationship.

Talks have not been abandoned, however. Existing nuclear arms limitation agreements still remain in force.

Secretary of State Cyrus R. Vance and Soviet Foreign Minister Andrei A. Gromyko have agreed to meet in May. U.S.-Soviet "working groups" are to be established to explore numerous other subjects ranging from halting all nuclear weapons tests to limiting military forces in the Indian Ocean.

But these are actions to help cushion the consequences of the blunt failure recorded tonight on the key subject of the Strategic Arms Limitation Talks (SALT).

The collapse of the strategic arms negotiations climaxes a remarkable two-month period in which the brandnew Carter administration boldly confronted the Soviets with criticism of Kremlin treatment of dissidents only to be told firmly that Moscow regarded such "interference" in its "internal affairs" as intolerable. The human-rights clash greatly disturbed the climate of Washington-Moscow relations and made the Kremlin suspicious of Carter's policies in general.

Besides the breakdown of the talks, another startling development today was the change in Soviet leader Leonid Brezhnev's appearance.

Brezhnev, his manner grim, his face puffy and ashen, his movements and his speech slurred, greeted Vance with barely a word. He refused to acknowledge reporters' questions, although he had always responded in the past. Gromyko called out sharply, "This is not a press conference."

Brezhnev's appearance was so startling that it immediately revived speculation about the 70-year-old leader's physical condition. His health may be a factor behind the determined Soviet drive for swift confirmation of the long-delayed SALT goals set by Brezhnev and former President Ford at Viadivostok in 1974. The Soviets have adamantly opposed the Carter administration's bid to supplement the Vladivostok accord with much more ambitious and lengthier negotiations for deeper cuts in arms levels.

It has been Brezhnev's great ambition to climax his career, Western analysts believe, with the signing of a SALT agreement at a summit meeting in Washington.

The dramatic failure to make headway with the United States on SALT could well damage Brezhnev's political standing with his Kremlin colleagues. He has championed the concept of superpower detente and has been personally identified with its successes and, increasingly in the recent past, with its failures.

Both nations tonight, in their immediate public comment on the impasse, sought to minimize the repercussions. What is bound to be most significant, however, is how the two sides privately interpret the consequences on their total relationship.

In a press conference tonight at Spaso House, the U.S. Embassy residence, Vance invoked a claim of "progress" to counter the atmosphere of despair.

"They were useful," Vance said drily of the three days of tense talks. "I think that U.S.-Soviet relations will continue to be good. I hope in the future we can strengthen those relations. Needless to say I am disappointed that we have failed to make progress in what I consider to be the most essential of all these areas, namely, the area of strategic nuclear arms, but I think our relations will continue to be good . . ."

A four-paragraph Soviet statement read on television tonight gave no indication of the Soviet mood. It said only that "The two sides agreed to continue their exchange of views "on SALT and other subjects."

While both sides still profess great interest in seeing relations develop, it seems likely that new approaches to East-West policy will be necessary if the present strained atmosphere is to be improved.

Soviet strategists have complained bitterly since March 24, when President Carter publicly crystalized his SALT objectives. They said the new administration was trying to change the game at the 11th hour with an inordinately complex new formula.

Soviet sources have also said they suspect U.S. trickery to shift the nuclear balance in American favor. The recent tempest of claims in the United States that there is a danger of approaching Soviet military superiority is the other side of the same coin.

Vance said tonight that the U.S. proposals here for arms limitations were "equitable," and he said that should be apparent.Vance said he was unable now, however, to discuss the key numbers involved and the specific terms of the U.S. offer.

The core of the problem is this: at Vladivostok in November 1974, Ford and Brezhnev agreed that the two nations should limit their intercontinental nuclear delivery systems - land and sea-based missiles and long-range bombers - to 2,400 on each side. Of this number, no more than 1,320 could carry multiple nuclear warheads.

That was hailed as a "ceiling" on the nuclear arms race, although the figure was immediately criticized as too high. The Soviet Union has more than 2,400 of these weapons and it would have to reduce existing forces; the United States has fewer and could increase them to reach the 2,400 ceiling. The Soviet missile launchers, however, are larger and can carry more payload, so the 2,400 level was regarded as a rounding-off number for parity.

Soon after Vladivostok, the most complicating issues arose, over two weapon system not discussed by Ford and Brezhnev; a new Soviet bomber known to NATO as Backfire, which the Soviets describe as a mediumrange bomber but which American officials claim has enough range to reach the United States; and on the opposide side, American long-range cruise missiles, now in the development stage, a more potent innovation.

Attempts to settle this issue reached a stalemate in 1976, although Brezhnev and former Secretary of State Henry A. Kissinger came near an agreement. The Ford administration, in an election year, backed away from the formula. That is basically the agreement the Soviet Union still wants and that, Vance said today the United States had rejected.

The proposal includes a trade-off on counting Backfire bombers and Cruise missiles but with sharp limitations on the numbers and range of Cruise missiles.

Carter, in the proposals that the Soviets have now rejected, offered two options: accepting the 2,400 Vladivostok levels, but deferring Backfire and cruise missiles with no restraints, or making "deep cuts" in the 2,400 Vladivostok ceiling with limitations on cruise missiles and on other nuclear weaponry.

The progress that Vance claimed tonight was agreement "to set up a number of U.S.-Soviet working groups in various areas to follow on the discussions" here.

As he listed them they are: "the area of comprehensive [total nuclear] test bans . . . chemical weapons . . . prior notification of missile test firing . . . anti-satellite weapons; the area of civil defense . . . possible military limitations in the Indian Ocean . . . radiological weapons . . .; the area of [curgs on sales or transfers] of conventional weapons" and "a regular schedule of meetings to deal with the whole question of proliferation."

Vance did not specify how these studies would differ from previous U.S. discussions on the subjects, but naming of "working groups" would signify that the United States and Soviet Union would now give more concentrated attention jointly to these issues.

The one prospect raised during the three days of talks for specific progress on a major point of U.S.-Soviet disagreement dissolved tonight.

Remarks by Gromyko Monday had aroused speculation about a possible significant departure in the Soviet position on terms for reconvening the Geneva conference on the Arab-Israeli conflict. He had indicated a possible shift on the issue of Palestinian participation.

Tonight Vance said Gromyko's remarks indicated no changed Soviet position.

On the fundamental issue here of nuclear arms controls, Vance said it is still possible to reach an agreement of some kind before the temporary limit on offensive strategic nuclear weapons set in 1972 expires on Oct. 3.

Asked whap happens next, Vance said, 'I would hope that they [the Soviet Union] would consider the proposals which we have made. We think they provide a reasonable basis for further discussion" despite the fact that the Soviets said that, "they did not find them acceptable."

Asked why the Soviet leaders rejected them, Vance remarked, "because they did not coincide with their view of what they thought was an equitable deal."

As for the Soviet proposal, Vance said, "the reason we found their proposal unacceptable is that it doesn't deal properly with the cruise missile issue."

Vance said he was "very disappointed," to be leaving here Thursday morning without achieving "a general framework" for new SALT negotiations. The only discussion continuing tonight, Vance said, was on the wording of a communique.

When asked if he believes that today's failure to agree on a SALT formula will result in acceleration of the arms race, Vance replied, "I would certainly hope not. I think that this would be a tragedy . . ."