For the second time in two months, Rhodesia's transitional government yesterday rejected a British-American proposal for a general peace conference that would include nationalist guerrilla leaders based outside Rhodesia.

Replying to a question from a black member of the white-dominated Parliament, Prime Minister Ian Smith said the new ruling four-man Executive Council "is not convinced that it would be in the national interest to delay the implementation of the Salisbury agreement by attending a conference which has no hope of success.

The Salisbury agreement of March 3 calls for election of a black majority government before the end of this year.

Smith said he and his three black colleagues on the council were not closing the door to continuing contacts with the two-Western powers.

"We are not opposed to going to another conference which for us will clearly be abortive."

At the same time, he renewed his appeal to the two externally based guerrilla leaders, Joshua Nkomo and Robert Mugabe, "to return in peace" and compete in elections provided for in the Salisbury agreement.

Smith denied reports that he was more inclined to attend a Western-sponsored all-parties conference than the council's three African nationalist leaders.

"We are all of the same opinion," he said.

It appears that the Anglo-American initiative to bring about a peaceful negotiated settlement to the 13-year-old Rhodesian constitutional dispute is virtually dead for the time being and perhaps forever. However, some local political analysts predicted that a conference might still be possible later in the summer if the transitional government had by then completely broken down.

The government is already in serious trouble and faced with a major crisis of confidence among both whites and blacks because of its failure to bring about a cease-fire in the guerrilla war.

The latest rejection of the Anglo-American peace proposal came as British envoy John Graham and U.S. Ambassador to Zambia Stephen Low returned for more talks with officials of the transitional government. However, since their arrival here Wednesday they have not met officially with any Executive Council members.

The two Western envoys spent 10 days here earlier this month vainly trying to convince the leaders of the transitional government to reverse their decision to boycott any Western-sponsored conference. Their own internal settlement agreement calls for election of a black majority government by Dec. 31.

Graham and Low met individually with the council's three black leaders - Bishop Abel Muzorewa, the Rev. Ndabaningi Sithole and Chief Jeremiah Chirau - but they have not seen Prime Minister Smith on either visit.

Smith's announcement yesterday came amid a rising chorus of black and white voices criticizing the performance of the interim government since the March 3 agreement, particularly its failure to achieve a cease-fire. Many of its initial supporters, both black and white, are now demanding that it attend the all-parties conference proposed by the United States and Britain.

Wednesday, 13 of the 15 African members of Parliament signed a statement saying it was now "of paramount national interest" that all of the country's black nationalist leaders get together "to save the lives of 7.5 million people," bring about an effective cease-fire, gain international recognition and "stop the involvement of Eastern and Western bloc powers in an international war over Rhodesia."

More than 9,000 African and European lives have already been lost in the war that began escalating in late 1971, and the death toll is now running at about 100 a week.

Prime Minister Smith said the Executive Council's renewed rejection of the Western invitation to an all-party conference was caused by the "obdurate attitude" of the Patriotic Front's two leaders. The front is an alliance of two guerrilla factions, Nkomo's based in Zambia and Mugabe's in Mozambique.

The two leaders have agreed to attend an all-party conference, but recently increased their demands and insisted that they must play a dominant role in the transition to black majority rule.

Smith charged in his statement yesterday that the front's leaders "have constantly demanded that they should take control of the country in both the civil and military spheres and that the security forces should virtually be disbanded."

Smith spoke before a racially divided Parliament whose 15 black members were reprimanded continually for harshly worded criticism of the white-led government. At one point, the speaker of the House, George Hartley, said any reference to the nationalist "Terrorists" as "freedom fighters or guerrillas" would be ruled out of order and expurgated from the official record.

The 15 black and 49 white members who make up the House of Assembly have shown remarkable unanimity in their criticism of the government during the debate, under way since Wednesday, on the state of affairs in Rhodesia.

Wednesday, the parliamentary caucus of the ruling white Rhodesian Front demanded "more action and less words" from the government's Executive Council, particularly in getting the nationalist guerrillas to agree to a cease-fire.

"The nation is waiting for the Executive Council to get off its collective posterior and to be seen implementing this agreement," said one white member.

Black members of Parliament, meanwhile, have been warning that there is no chance of holding elections if the war continues at its present level of violence and disruption in the tribal trust lands where the vast majority of the African population lives.