Prime Minister Ian Smith's Rhodesia Front Party swept back into power in today's elections with first forecasts indicating that it would win all 50 white seats in Parliament.

The early returns suggest that the white Rhodesian electorate, which numbers just over 92,000, may be moving toward a more moderate position.

The chief opposition, the new ultra-rightist Rhodesian Action Party, which ran on a platform of continued white domination, did so poorly that many candidates lost their campaign deposits of $162.

In an unexpected development, the small, liberal National Unifying Force, considered a fringe party here and fielding only 18 candidates, did better than the Rhodesian Action Party in many of the three-way races.

The overwhelming victory means that Smith now has the mandate he needs from the white minority of 270,000 to begin givig effect to the "internal option," a vaguely outlined settlement with moderate African nationalist leaders in a "broad-based" government of blacks and whites.

Smith's Rhodesia Front Party needed to win 44 seats to enable the prime minister to change the constitution.

First returns gave the party 35 seats. Still returns gave the party 35 seats. Still unreported were returns from rural areas, where Smith is strong, Smith's party held 38 seats before the election, which he called in an attempt to get a big enough bloc to make constitutional changes.

Although an Anglo-American team headed by British Foreign Secretary David Owen and Andrew Young, U.S. ambassador to the United Nations, expected here Thursday to put forward a Western-backed peace plan, most Rhodesians have acceped the probability that the proposals will be rejected. As the results were pouring in tonight, Smith furthered speculation that he will reject the plan. When asked by reporters what he would tell Owen and Young, he said, "I don't think I have to say anything to them. I believe the electorate has done it for me."

He described the victory of his party as "a mandate to get on the negotiate along the lines that we indicated we would," adding, "Rhodesians are not interested in anything else."

Smith's internal option is considered the only othe alternative to the Anglo-American plan for settling the troubled territory's 12-year-old constitutional crisis, even though there is little chance the outside world would recognize such a government.

The election results showed that whites have swung behind Smigh and his political settlement plans. His party's resounding win will be in his arsenal whan he meets Owen and Young. Smith has already rejected key points in their proposals and he said today that he has counter-proposals to put to the envoys. Moves toward an internal settlement with black moderates are reported to be more advanced than early in September.

Smith's immediate plans call for formation of a government that includes such moderate blacks as are willing to take part in the excercise. Among the main problems of the new government will be formulation of a constitution to be put to the white electorate in a referendum.

Now tht he has achieved his mandate, Smith faces the first challenge from the Anglo-American mission. He has already made it clear that he will reject the peace proposals if they include, as expected, Plans to disband the Rhodesian army, to force Smith's resignation in favor of a temporary British administrator general, and to allow nationalist guerrillas back into the country in a military capacity.

Although there are growing indications that Smith is finally willing to accept one-man, one-vote principle, which the Anglo-American plan calls for, it is certain tha he wants stronger gurantees for whites than the Western initiative calls for.

The prime minister announced on nationwide television last night that he will offer counter-proposals to the Anglo-American mission, although these are likely to be rejected in turn since the diplomatic team has indicated that there will be no further revisions.

Smith will have to move quickly if he rejects the plan. He promised during the campaign that his government, would complete the first steps of the internal settlement by Christmas.

Nationalist spokesmen for the country's two moderate leaders - Bishop Abel Muzorewa and the Rev. Noabaningi Sithole - have said they expect some approach from Smith requesting "consultative talks" by the end of September.

This will be Smith's last chance to end the old constitutional crisis and the five-year-old war. It is the first time he alone has taken the initiative and, although prospects look bleak for its acceptance by either nationalists or the outside world, the prime minister recognizes that the pressures from the rapidly escalating war may force him into unprecedented compromise to achieve a political settlement and avoid a military takeover by Marxist-backed guerrillas.

News services reported these other developments:

Owen and Young met in Kenya with Kenyan Vice President Daniel Arap Moi, Foreign Minister Munyua Waiyaki and William Eteki, secretary general of the Organization of African Unity.

Rhodesian nationalist leader Joshua Nkomo has rejected the idea of a U.N. peacekeeping force in Rhodesia during a transition to black-majority rule, the government-owned Zambia Daily Mail reported in Lusaka.

South African Prime Minister John Vorster was quoted by the West German magazine Quick as saying in an interview that he will not yield to pressure from any country, including the United States, to cooperate in pushing for a Rhodesian settlement.