Stirred by his recent tumultuous visit to South Africa, Sen. Edward M. Kennedy (D-Mass.) has vowed to force a Senate vote this year on proposed economic sanctions against that country over its apartheid policies.

Legislation to ban all new U.S. investment in South Africa passed the Democratic-majority House last year but died in the Republican-controlled Senate. It was described as contrary to the Reagan administration policy of "constructive engagement," which emphasizes behind-the-scenes diplomacy and quiet persuasion to get Pretoria to reform.

Many other senators on both sides of the aisle are also drawing up new approaches to South Africa that will range from stiff sanctions to symbolic gestures.

Sen. William Proxmire (D-Wis.) has introduced a package that would bar most new U.S. bank loans as well as new investment there and would also bar importation of South African gold krugerrands.

Sen. Nancy Landon Kassebaum (R-Kan.), who chairs the Foreign Relations subcommittee on Africa, has been cool to the idea of sanctions but favors measures to stimulate black-run businesses and stiffen equal-opportunity requirements for U.S. business there.

Sen. Charles McC. Mathias Jr. (R-Md.) has called on the western industrialized nations to add a South Africa item to the agenda for their June economic summit meeting in Bonn, possibly an endorsement of recent suggestions by Nobel Prize-winning Bishop Desmond Tutu.

Tutu advised U.S. firms to invest only on condition that the Pretoria government accomplish specific reforms within a certain period of time, on pain of withdrawing their projects.

"There's a lot going on already. Kennedy will have a hard time staking this out for himself," a key Senate staff aide said.

In a recent interview, Kennedy promised "to work with those already involved" for a renewed assault on apartheid to be proposed after Congress returns from its recess Feb. 18.

"One way or another the Senate will have an opportunity to vote on this this year. I can give you assurances on that," Kennedy said.

Kennedy and six members of his family spent eight emotional days in South Africa in early January, during which the senator was both criticized by government officials as he toured black residential areas and booed by small groups of militant black nationalists who denounced him as a capitalist tool.

The jeering blacks were a Marxist-Leninist group, Kennedy said, who were venting their frustration at their lives in South Africa by making no distinction between him and the powers that ruled them.

It was a striking contrast to the wild, adoring crowds that mobbed Kennedy's late brother Robert in a similar visit 20 years ago. That was after the U.S. civil rights movement had given oppressed peoples hope for change worldwide, Kennedy said.

Now, however, he said, he is appalled at the degree of polarization in South African society and convinced that U.S. policy is reinforcing a government doomed to crumble.

"South Africa is eventually going to be free. A continuation of existing policies will place the United States among the last allies of the basically white supremacist regime there and alienate a whole new generation of leaders," he said. "It is an awesome prospect.

Republicans have argued that withdrawing the $3 billion in direct U.S. investment in South Africa or curbing the estimated $4.5 billion in annual trade between the two countries would hurt most the 150,000 black employes of U.S. plants in South Africa. They also argue that sanctions would fail to work or would cause the government to become more intransigent.

"The overwhelming majority of blacks I talked to spoke of the violence in their lives already: the infant mortality, the forced relocations, the pass laws" that control workers' movements, Kennedy said. "They know they'll have to suffer even more, but they are prepared if it means a better life for their children." He said employes of U.S.-based firms were less than 1 percent of the work force.

Kennedy said that everywhere he went, members of the South African power structure wanted to know about the prospects for divestiture. "Their expressions on this were intense. They'd say that what the United States does makes no difference, but secondly, 'We hope the United States doesn't take any action,' " he said. "I drew the . . .conclusion that they would care very deeply and significantly" about any sanctions.