Early returns in the all-white elections here yesterday indicate that Prime Minister Pieter W. Botha's ruling National Party is retaining its majority strength in Parliament but with significant reductions in its popular vote.

Signaling their discontent with Botha's policies, many voters have either stayed away from the polls in protest or given their votes to the ultrarightist Reconstituted National Party or the moderate Progressive Federal Reform Party.

With more than one-third of the votes counted early today, the National Party had taken 51 percent of the popular vote, compared with 65 percent in 1977. The Progressive Federal Party had taken 25 percent and the Reconstituted National Party, best known by its Afrikaans name Herstigte, 8 percent. In 1977, they got 17 percent and 3 percent respectively.

The most significant gains so far have been made by the Progressive Party, which has so far taken five seats away from rival parties and in many constituencies reduced the National Party's majority by thousands of votes.

The most stunning victory for the Progressive Party was in the Cape Town suburb of Gardens, where its candidate defeated the National Party's Dawie De Villiers, a former ambassador to London and now minister of tourism. In 1977, the National Party won the Gardens constituency with a majority of 1,761 votes.

If the Progressive Party does not lose any of the seats it held before the election it will increase its seats in Parliament to at least 22. Botha's National Party controls 137 of the 165 seats in the current Parliament and is expected to maintain a similarly firm majority in the new legislature.

The Herstigte so far had not won any seats, but it was won more than 64,000 votes so far compared to its total support of 34,000 in 1977. The Herstigte candidates have also helped to decrease the popular votes of National Party candidates in several key constituencies.

It appeared possible that the Herstigte could win about 100,000 votes, which veteran Afrikaner political analyst Willem Kleynhans says would be a breakthrough. "It is quite likely that tens of thousands of voters, who are fed up with the P. W. Botha government, would see this as a good omen and flock to the Herstigte National Party ranks," Kleynhans said.

These early results indicate a decline in Botha's popularity and widespread discontent with his approach to making changes in South Africa's system of apartheid. Botha at first appeared about to make meaningful changes, but then backed down in face of protests within his own party.

On the one hand, the Herstigte accuses him of making concessions to blacks that will lead to black rule and endanger the white minority's political control. They have called him "the baldheaded Santa Claus of Africa."

On the other hand, the Progressive Federal Party says that any meaningful change requires moves far beyond what Botha is prepared to do. They are demanding a national convention of all races to discuss a new constitution.

About 2.3 million whites were eligible to vote in this all-white election to choose a government for the country's approximately 25 million people. Blacks are prohibited from voting by law.

One good example of National Party supporters' protests was the contest in which the generally popular Foreign Minister Pik Botha regained his seat, but with 2,000 fewer votes than in 1977. His Progressive Party opponent drew 1,000 more votes than in that election.

Most blacks were uninterested in the election, which will be the ninth victory for the National Party since it first came to power in 1948. Nthato Motlana, a black leader in the township of Soweto outside Johannesburg, expressed the feelings of most blacks when he commented that "our fight is not for P. W. Botha's half-baked reforms, which mean nothing at all. The next Parliament must be selected on the basis of one man one vote. That is our basic demand."