Pieter Botha, a strong supporter of the South African government's hard-line racial policies, was elected by the ruling National Party yesterday to succeed the ailing John Vorster as prime minister.

Botha, who has been defense minister for 12 years, immediately pledged to his party that "we will not bend our knees before Marxism or revolution."

The selection of Botha, 62, as South Africa's eighth prime minister means the continued dominance of conservatives in the National Party.

There is not expected to be any change, therefore, in the government's policy of apartheid, or racial segregation, nor in the slow pace at which nonpolitical forms of racial discrimination are being removed.

Moderates in the party who would like to see quicker elimination of racial discrimination were dealt a blow when their candidate, Foreign Minister Pike Botha, finished last in the three-way contest. The two candidates are not related.

In the first round of balloting by a caucus of the ruling party, which elects the prime minister, Pieter Botha recived 78 votes to 72 for Black Affairs Minister Connie Mulder and 22 for Pike Botha. With Pik Botha eliminated for second-round voting, Pieter Botha was elected handily, 98 to 74.

The National Party's 172-member parliamentary caucus, dominated by Afrikaners, also nominated Vorster, who resigned because of poor health, to be the party's candidate for the mainly ceremonial post of president. He is scheduled to be formally elected by the parliament today.

Botha, whose hawkish stance on security matters played a major role in South Africa's diplomatically disastrous involvement in the Angolan civil war in 1975-76, announced that he would continue as defense minister "for as long as I believe it suitable and practical." He said he did not intend making amy major Cabinet changes "in the near future."

The party's choice of Botha brings to power a man who in the past has resisted cooperating with the Western effort to seek a negotiated solution to the conflict between South Africa and the Soviet-armed guerrilla movement, the Southwest Africa People's Organization (SWAPO), in Namibia (Southwest Africa).

Botha is believed to be a strong supporter of the South African government's decision last week to back out of the Western-devised plan for U.N. supervised elections in Namibia and to hold its own elections there.

That decision has presented South Africa, and now Botha as the new prime minister, with one of its most severe international crises since the National Party came to power 30 years ago, bringing it the risk of further isolation from its traditional Western allies and the possibility of some form of economic sanctions from the United Nations.

In addition, Pretoria's decision to hold its own elections in Namibia is also expected to result in an escalation of SWAPO's guerrilla wr against the estimated 18,000 South African troops stationed there.

Speaking at a press conference only hours after his election, Botha said that South Africa's position on the elections "has made very clear," adding that "the door is always open for discussions."

Earlier, when asked about future relations between the United States and South Africa, Botha replied, "If they treat us as self-respecting country we will do the same with them." Relations between the two nations have deteriorated over the past two years with South Africa accusing Washington of meddling in its domestic affairs.

[In Washington, the State Department expressed hope yesterday tat the new Botha government will support the U.N. plan on Namibia's independence. Taking note of Botha's election, State Department spokesman Ken Brown said, "We wish him well as he assumes his new duties."]

Speaking to a crowd of about a thousand people from the steps of South Africa's parliament building, here, Botha pledged to work for improved realtions among races in South Africa, saying, "I believe there is enough common ground for us to work together to make this one of the most wonderful countries in the world."

Racial harmony in this segregated country of 4.5 million white and 18 million blacks is severely strained because of the government's refusal to abandon its policy of apartheid and its failure to press ahead rapidly to remove racial discrimination.

Botha, in militaristic fashion told his audience that South Africa was facing a "total onslaught" but "would not bend a knee to Marxism or revolution. We believe in the system of private initiative and will protect it as much as humanly possible."

South Africa had a "God-given task" to protect the Cape sea route, Botha said, voicing one of the most frequently express reasons for Western support of white-ruled South Africa as a strategic bulwark against Communist expansion.

Botha is generally regarded more as an able administrator than an intellectual. Under his aggressive stewardship in the past 12 years, the country's military strength has mushroomed and South Africa is even believed to have acuqired a nuclear weapon capability. Black-ruled African states interpret this buildup as a threat to them.

Botha was born in the Orange Free State - the heart of Afrikaner country.

After studying law, he joined the National Party as an organizer. His skill at building up an then using, the party machinery has propelled him up the political ladder.

He entered parliament 30 years ago when the National Party came to power and was appointed minister of community development and colored (mixed race) affairs in 1961.

While Botha has rarely expressed his views on issues affecting South Africa's black majority, he has often voiced somewhat moderate opinions on the position of the country's 2.6 million coloreds. He lives in Cape Province, where most of the coloreds live, and he was reponsible for opening up the first theater in Cape Town to coloreds.

Botha was Vorster's choice almost two years ago to head a special Cabinet committee that drafted a new constitutional plan for South Africa intended to give a measure of national political power to coloreds and to South Africa's 75,000 Indians.

Botha said yesterday he intend to proceed with the plan, although a large section of both the colored and Indian communities have rejected the plan on the grounds that it gives them no real political power.