While the Carter administration appears to have scored a major victory in its Middle East diplomacy, it seems perilously close to disaster in its carefully laid policy toward southern Africa.

South Africa's rejection of the Western peace plan for Namibia and Rhodesia's refusal to attend a preposed all-party conference have embarrassed the United States.

U.S. diplomats in the region have long been divided on whether the administration was overextending itself beyond its ability to shape the conflicts' outcome even while becoming a captive of African interests.

But having now become deeply involved - joining Britain in backing the conference proposal on Rhodesia and leading the Western diplomatic initiative on Namibia - Washington faces the consequences of its entrapment in the racially explosive politics of the region.

In both Rhodesia and Namibia, the time is fast approaching when the Carter administration may have to choose between being on the "white" or "black" side - or simply sit out the conflicts on the sidelines and thereby risk default to the Soviet Union and its allies.

The policy devised by such "Africanists" as ambassadors Andrew Young and Donald McHenry has aimed at avoiding such clear-cut choices while edging the United States toward far closer relationship with black Africa.

The main tactic has been to promote Western peace plans, in collaboration with the five front-line African states, that would seek to halt the natural drift of events in southern Africa toward race wars, block the expansion of Soviet influence and consolidate the good standing of the United States in black Africa.

The tactic came close to success before the Namibia accord fell apart, but has remained unworkable in Rhodesia. There the dynamics of the guerrilla war have continued to outpace various British-American peace initiatives for a negotiated settlement.

In retrospect, the fatal flaw in the administration's policy may prove to have been its awkward handling of South Africa, without whose support Washington could never have hoped to work its diplomacy in either Rhodesia or Namibia.

Washington's harsh criticism of that white-ruled country's racial policies and public pasturing in favor of black South Africans very quickly convinced the tough-minded Afrikaners of Pretoria that American policy was heavily against them. While the Carter administration was winning acclaim in black Africa for its bold new stance, its power of persvasion with South Africa was ebbing.

Now, with the election of the hardline Picter Botha as prime minister, Pretoria seems likely to be even more hostile to American diplomacy. Meanwhile, expectations in black Africa remain high, but so far unfulfilled, regarding what the United States ought to do.

Indeed, pressure from the front-line states is steadily building for London and Washington to take some dramatic action to implement their proposals for a revolution of the Rhodesian conflict, including the use of military force to remove Prime Minister Isa Smith and his white-minority establishment in Salisbury.

Zambian President Kenneth Kaunda is pressing Britain to call the all-party conference, including guerrillas and incumbents, as a way of halting the war that now threatens to spread to Mozambique and Zambia.

The war in Rhodesia has now reached a critical stage. The white-led biracial transitional government set up there last March is making a desparate appeal to Washington to save it from the Patriotic Front guerrillas, and their Soviet and black African supporters.

If Smith and his three black colleagues leading the interim Rhodesian government are allowed into the United States to take their case directly to the American public, they will argue that Washington owes them something.

Smith argues that Rhodesia's rebellious whites have now atoned for their 1965 political sign of unilaterally declaring their independence from Britain, just as did the United States some 200 years ago. He contends Rhodesia has more than met the terms set down by former secretary of state Henry Kissinger for American backing; the acceptance of black majority rule and the promise of elections based on universal suffrage.

But, he will say, his government never got any of the promised dividends, such as diplomatic recognition and Western economic assistance. Worse yet, he will argue, the West has decided to sell a moderate biracial government down the river to Communist-backed "Marxist terrorists."

The Rhodesian case is gaining support rapidly in Congress. But accepting it would necessarily lead the United States into one not of its own making and of limited direct national interest outside the present intense Soviet-American rivalry for influence in Africa.

Support of the Rhodesian transitional government would spell the end of the Carter administration's new Africa policy since Washington would lose credibility with its key black African partners - the five front-line states and Nigeria.

The Carter administration and British Labor government are now agonizing over what to do next to satisfy their black African allies, mindful that neither the British Parliament nor the American Congress is likely to approve armed intervention in Rhodesia against the whites while black Africa will not accept it in defense of them.

Just how the administration can extricate itself from this predicament and avoid losing the reservoir of good will it has built up in black Africa over the past two years is not easy to see.But it seems to have arrived at the point where it may have to make choices that will determine the true substance of its southern Africa policy.