The Reagan Administration, charging that South Africa's attacks against Botswana, Zambia and Zimbabwe "threatened the security of the region," yesterday ordered the expulsion of the senior South African defense attache in this country.

"We trust that this action will make clear to the South African government that the United States cannot tolerate disregard of the sovereignty of South Africa's neighbors," the State Department said in condemning commando raids in Botswana and Zimbabwe and the combined ground and air attacks against Zambia last Monday.

The order for Brigadier Alexander Potgeiter to leave the United States within 10 days came after Secretary of State George P. Shultz told Botswana's foreign minister, Gaositwe Chiepe, that the United States "shares an outrage" at the attacks.

"It was totally without justification and is completely unacceptable," a visibly angry Shultz said in a brief appearance before reporters.

He repeated those phrases several times and said the Pretoria government's actions were discouraging in light of U.S. efforts to bring about an easing of racial tensions in southern Africa.

His sense of discouragement was reflected in the subsequent department statement, which said:

"Our diplomacy for many years has aimed at stopping cross-border violence . . . . South Africa's resort to force has threatened the security of the region and violated the international principle that political avenues should be given every opportunity."

Shultz also harshly rejected South African President Pieter W. Botha's claim that Pretoria was defending itself against black African terrorists.

Botha had contended that South Africa acted in the same manner as did the United States when it bombed Libya last month in retaliation for what President Reagan claimed was Libyan support of anti-American terrorism.

"There is absolutely no comparison whatsoever, and the effort to link these things by the statements of the South African government are totally without foundation," Shultz replied.

He described Botswana as "a country we admire as a democracy" that has sought to resolve regional disputes through negotiation and where there is "a lack of any terrorist base."

Turning to Chiepe, he said he had accepted her explanation that the building complex hit by the South African commandos in Botswana had no connection with the African National Congress, which has been outlawed in South Africa as a terrorist organization. One Botswanan was killed in the attack.

The State Department announcement said that South Africa will not be permitted to replace Potgeiter "at this time" and that the top U.S. defense attache in Pretoria, Col. Robert Hastie, is being recalled to Washington for consultations.

Department officials said other personnel in the South African defense attache's office -- three military officers and two secretaries -- are being allowed to remain.

A spokesman for South African Ambassador Herbert Buekes said last night that the embassy would have no immediate comment on the U.S. order.

Despite his angry rhetoric, Shultz said the administration will continue its controversial "constructive engagement" policy. That involves trying to maintain friendship with the Botha government in order to promote an ending to apartheid and a negotiated settlement of its differences with black Africa.

When Shultz appeared with Chiepe, he said the administration was considering what actions it might take. But it was not immediately clear whether the defense attache's expulsion, announced several hours later, will be followed by other steps.

Department officials said earlier yesterday there are no plans to increase limited economic sanctions against South Africa because the administration thinks they would be ineffective and more harmful to the majority black population than to the white ruling class.

Shultz ruled out recalling U.S. Ambassador Herman Nickel, as was done last June to protest a raid on Botswana. He said that would make it more difficult "to keep working at ending apartheid . . . and trying to achieve a constructive result, discouraging though it may be."