Vice President Walter Mondale and South African Prime Minister John Vorster said separately today that they had made no progress toward resolving what Mondale called "fundamental and profound disagreement" on the need for change in South Africa's racial policies.

The two leaders held separate press conferences that clearly reflected the unyielding stance of both sides during two days of talks that ended this morning.

Mondale strongly warned South Africa and continuing racial discrimination and denials of political rights to blacks will bring a "worsening of relations" with the United States and increased violence in southern Africa.

For his part, Vorster told reporters that "a vital difference" between the two countries' positions "stems from the fact that knowingly or unknowingly the United States wants to equate the position of the American Negro with the South African black man."

This is an error, he continued, for "the black man in America has been divested of his African personality, language, identity," and "is an American in every sense of the word."

In South Africa, he said, blacks belong to tribal nations and cannot be absorbed politically into the white South African nation of 4 million.

Moving to end an eight-year period of increased American accommodation of the white-miority government by the Nixon and Ford administrations, MOndale delivered the blunt warning in eight hours of talks with Vorster.

"We hope that South Africans will not rely on any illusions that the U.S. will in the end intervene to save South Africa from the policies it is pursuing, for we will not do so." Mondale said shortly before leaving Vienna for Belgrade. "Every citizen should have the right to vote and every vote should be equally weighted," he said.

Projecting a strongly activist approach on southern Africa for the Carter administration, Mondale said the United States would "take actions based on our policy and to the detriment of the constructive relations we would prefer with South Africa" if there is no significant racial change there.

He said he had not listed for Vorster any specific actions the United States would take to express its disapproval, and he declined to discuss them with reporters.

Despite the stinging public criticism of his policies, Vorster endorsed American efforts to arrange a settlement to Rhodesia's guerrilla war and said that some progress has been made in negotiating a settlement in the disputed territory of Namibia (Southwest Africa).

Asked if the new American pressure on him would impede South Africa's efforts on Rhodesia and Nambia, Vorster sternly told newsmen that he "would not out of pique stop doing what I think is right."

Mondale announced that Vorster had in fact "agreed to support" a new British-American initiative in Rhodesia "to get the directly interested parties to agree to an independence constitution" and free elections "so that Zimbabwe can achieve independence during 1978, and peace."

Zimbabwe is the name for Rhodesia used by the African nationalists waging a guerrilla war against the white-settler government of Ian Smith, which depends ultimately for its survival on the South African government's willingness to let supplies cross its territory.

American officials asserted later that Mondale had succeeded in moving Vorster "into a somewhat more forward posture" on Rhodesia by getting him to agree to that the wording of Mondale's statement represented a joint position. Their view was that Vorster is more directly involved in the move for independence in Rhodesia now.

But Vorster, speaking immediately after Mondale in the ballroom of the Vienna Hilton hotel, did not use that language himself. Moreover, he said there had been no change in his goverment's general support for peace efforts in Rhodesia while refusing to interfere in internal affairs there.

Vorster also declined twice to respond directly to questions about Mondale's description of "encouraging" movement in Namibia, the mineral-rich territory South Africa occupies in defiance of United Nations demands for withdrawal.

Mondale said that the South Africans had agreed to free elections on a nationwide basis for a constituent assembly in Nambia, with United Nations involvement "in the electoral process to assure that it was fair and internationally acceptable." Sharp differences remain on interim steps to arrive at such an election, Monday acknowledged.

At the heart of the discussions, described by Mondale in two telephone calls Thursday to President Carter, was America's own experience with civil rights and racism particularly in Carter's native South.

Speaking crisply and frequently gesturing to explain and emphasize points, Mondale said he had come to convey to Vorster "the transformation in American society" that "effects not only our domestic life, but our foreign policy as well."

Mondale has stressed the Carter administration's "commitment to human rights" at each stop of his five-nation tour in Europe, which is to end Sunday with a visit to Britain.

Vorster spoke for an hour, rarely changing expression or tone as he recited standard official defenses of apartheid and spoke of progress in permitting multiracial sport, providing job opportunities and education for South Africa's 16 million blacks, and setting up tribal homelands where blacks can vote for local leaders.

Vorster had made it clear, Mondale said, that the South Africans do not intend to make any changes as a result of the talks. But the South African leader gave a small hint that he had not given up on eventually finding some common ground with the Carter administration.