The United States called its ambassador to South Africa home yesterday to protest a South African raid into neighboring Botswana and other actions that threaten to upset U.S. diplomatic strategy in the region.

State Department spokesman Bernard Kalb said U.S. Ambassador Herman W. Nickel was being brought back for consultations "to review the situation." He gave no indication how long Nickel would remain here.

It is the first time the Reagan administration has taken such a step against South Africa. The diplomatic slap is the strongest demonstration yet of growing U.S. anger with a number of recent South African actions, including an abortive raid on a U.S.-operated oil installation in northern Angola last month.

The raids in Botswana and Angola, plus Pretoria's decision to set up a white-dominated "transitional government" in Namibia, which South Africa administers in defiance of a United Nations resolution, have brought a halt to U.S.-led negotiations on Namibian independence and on the withdrawal of Cuban troops from Angola.

The raids also have called into serious question the effectiveness of the administration's policy of "constructive engagement" toward South Africa, with even conservative Republicans in Congress branding the nonconfrontational policy a failure and calling for economic sanctions.

Kalb said the United States "rejects categorically" the South African policy of mounting raids into neighboring black countries for suspected guerrillas. The latest attack on Botswana, he said, was particularly deplorable because talks were already under way between the two governments to control cross-border violence.

At least 16 persons were reported killed when South African commandos raided 10 alleged nationalist guerrilla homes and offices in Botswana's capital, Gaborone, early Friday.

Kalb said the Botswana raid and the May 21 attempt to blow up the partly U.S.-owned Cabinda Gulf Oil Company facility in Angola raised "the most serious questions" about South Africa's recent conduct and policy.

"Actions have been taken that placed at physical risk U.S. lives and property," he said, adding that Pretoria still has not satisfactorily explained the Angolan raid.

Deputy Assistant Secretary of State Frank G. Wisner told the Senate Foreign Relations Committee earlier that the State Department had summoned South African Ambassador Bernardus G. Fourie seven times for a better explanation of the incident, but still was not satisfied.

"Respect for the national sovereignty of all states and the inviolability of international borders are key principles in international relations, and no state can arrogate to itself the right to violate these principles," Kalb said in a prepared statement.

"We cannot and will not condone violations of these principles. Such cross-border violence only complicates efforts to bring peace to the South African region," he added.

Despite these expressions of U.S. displeasure, the State Department also said that it did not plan to alter or abandon its battered "constructive engagement" policy -- the use of quiet diplomacy rather than pressure such as economic sanctions to force the country to end its policy of strict racial separation.

"We continue to believe that the policy that we have pursued consistently in recent years is the only policy that offers the prospect of advancing our goals of regional peace and security and constructive change in South Africa," another spokesman said. "We will continue to pursue those goals."