Former prime minister Ian Smith's Republican Front Party won an election tonight for a Salisbury parliamentary seat reserved for whites, defeating a white splinter party that had campaigned on a platform of increased cooperation with the black-ruled government.

The government of Prime Minister Robert Mugabe had closely followed this election and an earlier one also won by Smith's party as an indication of white attitude toward its policy of reconciliation. Any subsequent hardening of racial lines could lead to increased white departures.

John Probert, the Republican Front candidate, received 1,202 votes in the wealthy Salisbury suburb of Borrowdale, while Chris Mercer of the newly formed Democratic Party got 594 votes.

Only 29 percent of those registered voted, an indication of white apathy since black-majority rule was established last year.

The victory demonstrated Smith's continuing hold on the fewer than 200,000 whites who still control the economy in this nation of 7 million people. The party, which until last month was known as the Rhodesian Front after the former name for Zimbabwe, has never lost a white seat in Parliament since Smith gained power 17 years ago.

Until two years ago the victories meant that Smith, who declared independence from Britain illegally in 1965 to maintain white-minority rule, continued to lead the country in a bloody war against guerrillas led by Mugabe.

These days the victories are somewhat hollow since the 3 percent white minority can contest only 20 reserved seats out of the 100 in Parliament and thus have no chance to gain power.

On taking office last year, Mugabe proclaimed a policy of reconciliation aimed at encouraging whites to remain in the country. So far, the policy has been relatively successful with only about 24,000 whites leaving in the first 14 months of black government despite a bitter seven-year war to prevent majority rule.

In recent months, however, Mugabe's ruling party has increasingly attacked the Republican Front as racist and obstructionist.

A few moderate whites have joined Bugabe's party but politics in Zimbabwe are still divided along racial lines. To give whites another choice, Andre Holland, a Republican Front member of Parliament, quit the party and formed the Democratic Party three months ago.

Holland lost a by-election two weeks ago almost a two-to-one margin. Coupled with tonight's landslide loss, it appears that his party's effort to contest Smith, with the tacit support of Mugabe, is doomed.

Latent racial tension here was evident in a parliamentary debate on a health bill this month. As the white officials walked out of the debate, outspoken Health Minister Herbert Ushewokunze called them "twits."

Last week Smith engaged in a public mud-slinging match with Finance Minister Enos Nkala in the press. Nkala said he was "fed up" with Smith's "alarmist" statements about runaway inflation and threatened the former prime minister with detention or deportation.

Smith is "undermining the policy of reconciliation," Nkala said. "We want to live with them [the whites], but they have to realize reconciliation is a two-way process. Whites must respect blacks."

Smith, who imprisoned Mugabe and other nationalists leaders for 10 years or more, responded by repeating his charges that Nkala was mishandling the economy and accused him of being "not only incompetent but a tyrant into the bargain."

Smith was at his vintage best last night at an election-eve rally in which he enraptured his all-white audience of about 150.

Judging by the applause, the farmer-politician told his supporters what they wanted to hear. Careful not to criticize Mugabe, whom many whites have come to admire, Smith attacked the government's economic and foreign policies.

The government, he said, "spends money like it is water coming out of a tap" and he warned that unless the policy changes the inflation rate will hit 30 percent. Officials figures place the rate at less than 10 percent but there is concern over future trends and budget deficit of more than $600 million.

He praised President Reagan's policy of friendship toward South Africa and called him "the savior of the free world."