Secretary of State George P. Shultz continued efforts yesterday to dampen growing congressional sentiment for passage of tough economic sanctions against South Africa.

Shultz, who discussed South Africa with Senate Majority Leader Robert J. Dole (R-Kan.) on Tuesday, met at the State Department yesterday with Foreign Relations Committee Chairman Richard G. Lugar (R-Ind.).

Lugar said afterward that he hoped President Reagan will speak publicly on the South African issue before the committee begins three days of hearings on it next Tuesday. Lugar also said he hoped Reagan would settle on "a course of action" to respond to the surge in violence in South Africa.

White House officials said Reagan probably will speak on the issue early next week in connection with the expected official announcement of his appointment of Robert J. Brown, a black businessman from North Carolina, to be the next U.S. ambassador to Pretoria.

Brown, 51, was at the State Department yesterday, and told reporters he would accept the ambassadorship if it is offered. "I hope and pray that I could" influence the situation in South Africa, Brown said.

Though Reagan remains opposed to the imposition of sweeping sanctions against South Africa, his administration is faced with strong momentum in Congress for passage of sanctions legislation. Administration policy toward South Africa could be a key issue in several close Senate races in November.

The administration appears to be seeking a solution that would satisfy these political concerns in Congress but also head off stringent legislation.

A Senate GOP source said that much will depend on whether any new steps against South Africa that Reagan announces are seen in Congress as a strong enough U.S. response.

"We're convinced that there will be some kind of vote on sanctions on the Senate floor," he said. "It will be almost impossible to stop. The question is will the administration's efforts be seen as strong enough to turn it toward a more moderate approach?"

The official said that South Africa "hits home more than any other foreign policy issue" and that in a number of close Senate races "a few percentage points vote by blacks could make the difference."

"A lot of senators will want to go on record against South Africa before the election," he added. "If the president doesn't give them enough perceptual cover, they will want to do more."

The House last month approved sweeping sanctions against South Africa, including a total trade embargo and a requirement that U.S. firms withdraw their investments from the country. That legislation, sponsored by Rep. Ronald V. Dellums (D-Calif.), was unexpectedly passed as a substitute for a milder bill sponsored by Rep. William H. Gray III (D-Pa.). The Gray measure would ban new investments and require the U.S. computer industry to withdraw if Pretoria does not free all political prisoners and begin negotiations to dismantle the apartheid system within 18 months.

In the Senate, legislation similar to the Gray measure is being sponsored by Sens. Edward M. Kennedy (D-Mass.) and Lowell P. Weicker Jr. (R-Conn.). Sen. Nancy Landon Kassebaum (R-Kan.) has introduced a bill that would impose an escalating series of sanctions tied to Pretoria's response.

As a result of the House passage of the far reaching Dellums measure, many in the Senate expect to pass a sanctions bill too.

"It looks very good for some strong legislation," said Minority Whip Alan Cranston (D-Calif.).

The expected appointment of a black ambassador was seen as part of the administration's attempt to cool growing congressional enthusiasm for sanctions. But while Brown has been praised by both liberals and conservatives as a good choice for the job, warnings surfaced yesterday that his appointment would be wasted if it is not accompanied by a change in policy.

Gray said Brown would succeed as ambassador "if he is given the ability to implement a totally new policy." Under existing policy, Gray added, Brown "will send the worst possible message to black South Africans."

Civil rights leader Jesse L. Jackson, who has supported Brown's expected appointment, said, "If it's a signal for change, it's welcome. If it's a sideshow and a diversion, it will stimulate the protest" against administration policy.