The United States and Britain obtained agreement yesterday from Rhodesia's temporary government to consider new peace talks with black guerrilla leaders of the Patriotic Front.

Secretary of State Cyrus Vance and British Foreign Secretary David Owen advised the transitional rulers of Rhodesia that they will cross a bridge of no return unless they reconsider their rejection last week of an all-parties conference on Rhodesia's future.

The British and American teams ended their special mission to Salisbury hoping they have bought time for another round of diplomacy seeking to head off a black-against-black conflict that could spread across southern Africa.

Vance and Owen asked for neither a yes nor no response now from Prime Minister Ian Smith and the three black moderate leaders who joined forces last month in the name of Rhodesia's executive council. As a result, there was no conclusive outcome to the unusual top-level Anglo-American mission to this former British colony.

"Think about it hard," was the message left with Smith, Bishop Abel Muzorewa, the Rev. Ndabaningi [WORD ILLEGIBLE] and Chief Jeremiah Chirau. The Vance-Owen team urged the group to take some weeks to make a [WORD ILLEGIBLE] response.

The envoys hope this interval will provide a cooling-off period during which the rival internal and external factions will contemplate a compromise on who will rule an independent Rhodesia under control of a black majority.

When reporters asked Smith after the meeting if Vance and Owen had convinced him of the benefits of an all-parties conference, he replied, "I doubt it!"

He added, however, that the Anglo-American proposals would be given "mature and good consideration."

"We gave them some food for thought and in return they gave us some food for thought," he said.

Vance's visit here for a few hours of bargaining was the first by an American secretary of state to Rhodesia. Since Rhodesia's whites unilaterally declared independence from British colonial rule in 1965, incurring the wrath of black Africa and U.N. condemnation and sanctions. Rhodesia has been considered off limits for any such visit.

Vance, concluding that the threat of wider war was overriding, told reporters as he arrived from South Africa that "I was prepared to go anywhere to try to talk to the parties" to keep open any hope for a diplomatic settlement.

By setting their diplomatic expectations so low, Vance and Owen also sought to preclude the damage that a double setback within three days would cause to the controversial Anglo-American formula to bring one-man, one-vote democracy to this beleaguered nation of 6.7 million blacks and 263,000 whites.

Reporters accompanying Vance were told he would consider anything short of an outright rejection of an all-parties conference to be an encouraging sign.

A crowd of black supporters of Bishop Muzorewa demonstrated outside the building where the meeting was held. They carried signs critical of the Anglo-American position, saying, for example, "Owen and Vance, be realistic," and "Vance and Owen, concede defeat."

The protests indicated the extent of change in Rhodesia, since it was the first time in the memory of observers here that blacks demonstrated in support of Smith, who headed a white minority government until last month.

The external nationalist forces, the Patriotic Front led by joshua Nkomo and Robert Mugabe, dealt their own rebuff last week to the Anglo-American formula for peace in Rhodesia, which Smith had scorned at the outset. The Front suddenly raised its price for a settlement, claiming that its gains on the battlefield in the six-year old guerrilla war against Smith's white-minority government now entitle it to "a predominant role" during the transition to free elections and majority rule.

Owen, arriving in Rhodesia aboard Vance's Air Force jet, said that "if we just sit back," the conflict will spread to the bordering nations of Zambia and Mozambique where guerrilla forces are based.

Owen noted that the competing black nationalists inside and outside Rhodesia have often been allied in the past and have "shifted their alliances" but "they've never fought each other actually in armed conflict." Therefore, Owen said, a reshaping of alliances cannot be foreclosed and no possibility for encouraging that option should be ignored.

Owen recalled that only a year ago, when he met Smith in Salisbury, the possibility for one-man, one-vote rights in Rhodesia "was totally ruled out."

Even now, Owen said, "everybody's in a negotiating position" demanding extreme terms for a compromise.

"It isn't our job to pick and choose between differing and argued-over settlements," said Owen, but only to set out principles for fair elections and democratic rule. Neither side, he said, can produce on its own the "crucial" requisite - "to hold elections in an atmosphere of cease-fire."

Significantly, Owen said, even South Africa, Rhodesia's closest ally, was "not saying to us, pack it in, support the internal settlement . . . I thought they felt it was understandable that we should go on trying to bring about a meeting of minds, building on areas of agreement, trying to bring people together, and that's our task."