Secretary of State Cyrus R. Vance called on South Africa yesterday to dismantle its apartheid system or face inevitable deterioration of its relations with the United States.

In the first comprehensive statement of the Carter administration's policies toward Africa, Vance declared, "We cannot defend a government that is based on a system of racial domination and remain true to ourselves."

Vnace proposed no timetable or detailes plan for South Africa to follow, and said its form of government is a matter for that country to decide. But he said progress must be made soon toward "an end to racial discrimination and the establishment of a new course toward full political participation by all South Africans" if peaceful change in that country is ever to be possible.

The forum for Vance'ss address in St. Louis - the annual convention of the National Association for the Advancement of Colored People - suggested an effort by the administration to align its Africa policies with the views of U.S. blacks and to gain their active support. His speech won frequent applause, most notably and enthusiastically when he praised U.N. Ambassador Andrew Young.

As outlined by Vance, the Carter policies depart in two fundamental respects from those of former Secretary of State Henry A. Kissinger and the Ford administration.

First, there is greater emphasis on opposition to apartheid in SOuth African even while the United States seeks South African cooperation in arranging movement toward majority rule in Rhodesia and Namibia. Vance said the decision to move against apartheid was taken "after careful consideration," on the ground that the problems of Southern Africa are intertwined. He maintained it "would be wrong and would not work" to ignore the racial problem of South Africa.

Second, there is an explicit move away from what Vance termed "a negative, reactive American policy that seeks only to oppose Soviet or Cuban involvement." The Communist threat was considerated to be the motive force behind the Africa policy of Kissinger, who became keenly interested in the continent only after Cuban intervention in Angola. Angola was not mentioned in Vnace's addredd.

Far from muting these changes in policy, Vance's address went out of its way to call attention to them. The turns in direction and emphasis are likely to win the approval of black africa and blacks at home, but may cause a reaction from African whites and U.S. conservatives.

The first reaction from South African foreign ministry official in Johannesburg called the speech "encouraging" because it welcomed positive action by South Africa and did not mention "one man, one vote," a phrase sometime employed by Vice President Mondale in speaking of South Africa's future.

Senior U.S. officials told reporters that ommission of thet phrase was not significant and that the policy as stated by Vance did not represent any weakening of what other officials have said before.

While saying that anti-communism is not a sufficient basis for U.S. policy in Africa. Vnace also delcared that "we cannot ignore . . . and we oppose" the increase in Soviet arms and Cuban personnel there.

Vance also declared. "We will consider sympathetically appeals for assistance from states which are threatened by a buildup of foreign military equipment and advisers on their borders." He cited the Horn of Africa, where the United States is actively considering a substantial economic aid program for Somalia in seeking to woo it away from Soviet influence. The possibility of military sales to Somalia is also being discussed within the government.

In another area of East West competition; the United States has begun a military supply relationship with the Sudan, which is ousting its Soviet advisers and turning toward the West. Informed sources said Washington iss likely to authorize the supply of jet fighters to Sudan in the near future.

Speaking of the recent invasion of Zaire by forces based in Angola. Vance said "we see no advantage in unilateral response and emphasizing their East-West implications. We prefer to work with African nations - and with our European allies - in positive efforts to resolve such disputes."

The United States supplied only "nonlethal" aid to Zaire during its challenge from Shaba insurgents. But Morocco sent troops and France supplied transportation weapons and other aid in a joint effort that appears to have succeeded.

Vance emphasized the administration's commitment to human rights at several points in the address. "One of the most significant events of modern African history," he said, is the recent decision by British COmmonwealth countries, including some from Africa ot condemn the "massive violation of human rights in Uganda."

U.S. officials have had little to say about Uganda since President Idi Amin seemed to threaten U.S. citizens living there in reaction to a public condemnation of his rule by President Carter.

An "affirmative" U.S. policy toward Africa must seek to foster prosperity through greater U.S. partricipation in African development and U.S. efforts to help Africans resolve their own disputes.Vance said. He also declared that U.S. policies "should recognized and encourage African nationalism."

According to Vance, U.S. economic assistance to the continent is being increased from $271 million in fiscal year 1978. He called on European countries to increase their aid to African also, and said the United States would seek to coordinate the efforts.