The Carter administration's pursuit of peaceful solutions to southern Africa's racial conflicts suffered another blow yesterday when South Africa spurned U.S. proposals to keep the United Nations plan for Namibia's independence from collapse.

U.S. sources said that South Africa, in a long-awaited reply to American proposals for ironing out disputes over Namibia, declared its intention to proceed with a go-it-alone plan for Namibia elections that would exclude the guerrilla forces of the Southwest Africa People's Organization (SWAPO).

According to the sources, South Africa also said it will not cooperate with the U.N. plan unless steps are taken to prevent SWAPO from establishing military bases inside Namibia and to monitor the activities of SWAPO bases in neighboring black African countries.

The sources stressed that Prime Minister Pieter W. Botha's Pretoria government stopped short of saying it will pull out of the plan for U.N.-supervised elections in the territory it has controlled since shortly after World War I.

But they also conceded that the response made by Pretoria yesterday seemingly has put it on a new collision course with the United States and its western allies that could force them to consider economic sanctions against South Africa.

That was confirmed indirectly by Deputy U.N. Ambassador Donald McHenry, who told a congressional committee yesterday that collapse of the Namibia agreement could destroy the prospects for negotiated settlement of the more serious conflict in Rhodesia.

The administration is under heavy pressure in Congress to recognize the results of the recent elections in Rhodesia and lift sanctions against the regime there. These pressures yesterday brought Secretary of State Cyrus R. Vance to Capitol Hill to caution against precipitate action and urge that President Carter be given time to follow a congressional mandate on deciding whether to recognize the new government there.

The Namibia agreement bears directly on the Rhodesian question because the United States, which took the lead in negotiating the plan, has held it up as a model demonstrating that conflicts between black majorities and white minorities in southern Africa can be settled without blloodshed.

For more than a decade, the sparsely populated territory on South Africa's northwest border has been the scene of sporadic warfare between South African forces and SWAPO, which the Pretoria government regards as communist-controlled.

Although both sides agreed to the independence plan voted by the U.N.. Security Council last July, South Africa since has insisted on pursuing a separate track of internal elections in which SWAPO refuses to participate.Washington and its allies have agreed to acquiesce tacitly in these elections, while refusing to recognize their results and insisting that implementation of U.N.-supervised elections should continue.

For a time, South Africa cooperated with that approach. Several weeks ago, though, it halted its cooperation, charging that SWAPO's unfettered activities within and without Namibia were allowing it to intimidate voters and gain an unfair advantage over Namibia's other political parties.

McHenry, testifying before a House subcommittee on Africa, said the United States and its allies had tried to overcome South Africa's objections with proposals that SWAPO bases set up in Namibia be located away from population centers and be under U.N. monitoring. He also said the governments of neighboring Angola and Zambia had promised to ensure that SWAPO bases within their borders would not be used to disrupt a cease-fire.

However, the sources said, South Africa's reply rejected these steps as inadequate. They added that Pretoria signaled its intention to continue its go-it-alone, separate election plan with contests among the other Namibia parties for a national assembly and an executive council.

The sources said it was not clear whether South Africa is heading for a definitive break with the U.N. plan or stalling for time in which to see whether SWAPO can be circumscribed sufficiently to keep it from winning power under U.N.-supervised elections.

The latest move by Pretoria, they added, appears to be dictated by continuing hostility within South Africa to the U.N. plan and by the Botha government's vulnerability to a domestic influence-buying scandal.

In addition, the sources said, Pretoria may have been emboldened to take a tough line by last week's election of Conservative Prime Minister Margaret Thatcher in Britain.

She has indicated that her government might scrap Britain's joint plan with the United States to press for a Rhodesia solution including that country's guerrillas and instead recognize the incoming black majority government of Bishop Abel Muzorewa.

That possibility has raised considerable behind-the-scenes concern in the Carter administration, and Vance, in his talks on Capitol Hill yesterday, revealed that he will go to London on May 20 for urgent consultations with the new British foreign secretary, Lord Carrington.

But, the sources said, as long as the new British government's position toward southern Africa remains uncertain, the Botha government appears to have decided to hold back on Namibia in hopes that Britain could prove a chink in the West's solid front of support for the U.N. plan.

Vance went to Capitol Hill at the invitation of four senators-Jesse Helms (R-N.C.), S.I. Hayakawa (R-Calif.), Richard S. Schweiker (R-Pa.) and Dennis DeConcini (D-Ariz.)-who have been associated with efforts to bring about an immediate lifting of sanctions against Rhodesia.

Vance's efforts appeared, for the moment at least, to have warded off an immediate confrontation. Helms, who earlier had promised to introduce sanctions-lifting legislation this week, said afterward that he didn't want a head-on collision with the administration and would consult further with Vance.

In the meeting Vance noted that, under legislation passed by Congress last year, Carter has to make a determination on whether the Rhodesian elections were fair and whether sanctions should be lifted. He pledged that Carter would do so within two weeks after the Muzorewa government, which is scheduled to take office at the end of the month, is installed, and he urged that the senators give the president the necessary time. CAPTION:

Picture, Secretary of State Cyrus R. Vance discussing the situation in southern Africa yesterday on Capitol Hill with, from left, Sens. S. I. Hayakawa (R-Calif.), Dennis DeConcini (D-Ariz.) and Jesse Helms (R-N.C.). AP