An extraordinary series of four special elections in South Africa has ended with what one leading member of Parliament here described today as "a slight setback for reform, but not the end of it."

The extreme rightist Conservative Party, formed by a group that split from the ruling National Party last year in protest against its plans for token reforms in the country's segregationist system known as apartheid, cut deeply into the government's support but failed to cause an upset.

The Conservatives are no immediate threat to the government's hold on power, but the results show them to have significant support among Afrikaner farmers and lower income workers.

These are the two categories of whites who made up the original constituency that carried the National Party to power 35 years ago. Some observers think this may alarm the government and make it even more tentative in its already cautious attitude to reform.

The other feature of the special elections was that the liberal Progressive Federal Party, which had hoped to prod the government into extending its reforms by snatching a seat from it in the capital city of Pretoria, failed to do so. The National Party retained the seat.

There were no immediate signs that the government might abandon the reformist course that Prime Minister P. W. Botha began charting when he came to power four years ago.

Western diplomats were relieved to hear Foreign Minister Roelof F. Botha give an assurance in Parliament this afternoon that the government would not deviate from its commitment.

Prime Minister Botha sounded more circumspect. In a statement after the results were announced, he said they showed the electorate to be deeply divided and he pledged to study the implications of this.

It was an anxious day for members of the government today as they waited for recounts in the last and most critical of the four special elections.

The fourth result, in which the government just scraped by, involved the man who precipitated the extraordinary series of elections, Stephanus P. Botha, minister of manpower.

In a headstrong moment in February, Botha threw a challenge across the floor of Parliament to the Conservative leader, Andries P. Treurnicht, suggesting they both resign their seats to test their support at reelection contests. Botha also stipulated that Thomas Langley, a Conservative from Pretoria, should be his opponent.

The Conservatives accepted the challenge and all three men resigned.

Coincidentally, a member of the Transvaal provincial council died on the same day the challenges were issued. This required another special election in the gold-mining town of Carletonville.

Treurnicht won a handsome victory, improving on his winning margin of 1981 when he was a government minister and National Party leader in Transvaal Province.

He did this even though the rightist vote was split between his party and another, the Herstigte Nasionale party. Together these two parties polled 69 percent of the votes cast in that constituency, traditionally a safe one for the National Party.

By comparison, Stephanus Botha's win was embarrassingly narrow. After 17 hours of counting and recounting, he was declared the winner by a margin of 621 votes. That was 3,000 less than his winning margin in 1981, a dramatic reduction for South Africa's small, whites-only voters' rolls.

Although the National Party won the Carletonville seat comfortably, it was with fewer votes than the combined totals of the two rightist parties.

Conservatives said reaching an election pact with the Herstigtes would now be a priority.