Secretary of State Cyrus Vance and Australian Prime Minister Malcolm Fraser today agreed that the constitution of Zimbabwe-Rhodesia must be modified before their countries would recognize the new government of the breakaway British colony.

The agreement is a sharp setback to British Prime Minister Margaret Thatcher, who had left here less than 48 hours before Vance arrived.

In her Australian talks and public comments, Thatcher had praised the constitution for providing majority rule, a black leader and protection for the minority whites. She had talked not in terms of "if" but "when" the new government would be recognized by Britain -- and other Commonwealth nations such as Australia.

Officials said after the Vance-Fraser meeting that two conditions had to be met before Australia and the United States would recognized an independent Zimbawe-Rodhesia.

One, the two officials agreed, is changes to constitutional provisions that guarantee the 4 percent of whites in the country's population a position of special power for at least 10 years and leave whites in command of all key positions in the military and civil service.

The other condition is "adequate" endorsement of the constitutional changes from black African countries.

The decision of the two leaders to make public their agreement on policy for Zimbawe-Rhodesia was especially significant because of the role Fraser is expected to play at next month's Commonwealth summit meeting in Lusaka, Zambia.

Despite his policy differences with Britain, Fraser agreed with Thatcher at the weekend to play a role in Lusaka as a moderator between the old White Commonwealth members and the largely poor non-white nations that now make up the vast majority of the 41-member Commonwealth.

Vance's talks with Fraser also included U.S. Ambassador-at-large Dick Clarke. Vance is also here to take part in the council meeting of ANZUS, the Australia, New Zealand and U.S. security pact.

Clark's presence emphasized Australian concern over the Vietnamese refugee problem.

U.S. officials left no doubt that a sizeable increase in Australia's present intake of 10,5000 refugees a year was part of the Vance-Clark mission here.

Although ANZUS was never invoked as justification for the Vietnamese intervention, Australia was a military participant alongside tthe United States. But Australian opinion polls show that the overwhemingly Anglo-Saxon Australian people are about 70 percent opposed to admitting any more Vietnamese refugees.

Vance and Australian Foreign Minister Andrew Peacock are expected to sign a communique Thursday endorsing a more active American naval presence in the Indian Ocean. They are also expected to sign a new nuclear safeguards agreements as a prelude to greater sales of Australian uranium to the United States for power production.