Irving Kristol yesterday urged businessmen and bankers here not to yield to demands that they boycotted South Africa, Chile, the Soviet Union, or Yugoslavia in an effort to demonstrate "corporate social responsibility."

The New York educator and author was addressing the International Monetary Conference sponsored by the American Bankers Association. He said that some businesses and banks, "either out of mindlessness or cowardice, have capitulated to some such demands, thinking that this will be the end of the matter."

Instead, efforts to demonstrate corporate social responsibilities are a form of appeasement as useless as the concessions made by universities in the 1960s in an effort to buy campus peace, Kristol said.

But he was vigorously challenged from the floor by several bankers who said they could not disregard social responsibilities. Among those taking strong issue with Kristol were Christian Karsten, chairman of the board of the Amsterdam-Rotterdam and John Perkins, president of the Continental Illinois National Bank and president-elect of the ABA.

Kristol's hard-nosed advice to bankers to concern themselves primarily with keeping their banks solvent coincided with a speech here yesterday by Harry Oppenheimer, chairman of the Anglo-American Corp. of Johannesburg. Oppenheimer said that those who seek to change South Africa's racial policies by cutting that country off from the world's capital markets "are aiming at change by violence."

Kristol's thesis was at odds with a suggestion made at the opening session here by former secretary of state Henry Kissinger that political conditions be attached to economic aid to socialist countries in an effort to slow Soviet military aggression.

Oppenheimer, who heads the leading financial holding company of South Africa, opposes the government's apartheid policy on economic grounds. He said that it is "naive in the extreme" to think that the government party's views could be changed by an attempt to isolate South Africa.

The code of employment practices drawn up by the Rev. Leon Sullivan of the Zion Baptist Church of Philadelphia, which called for immediate establishment of minimum wages for blacks in South Africa, ignores reality, Oppenheimer said.

Oppenheimer told the bankers he supports the principle behind the Sullivan code "but it is plain from the figures that the possibility of improving living standards of the black people to an acceptable level depends not so much on a different distribution of the national income, as on a much more rapid economic growth than can in the present circumstances be envisaged."

The South African business leader said in his opinion "sound business judgment" by the bankers could in the end do more for black South African freedom than "anything likely to be achieved by the incongruous alliance of church groups and communist-armed freedom fighters."

Few of the 120 large international bankers who attended this informal but highly influential annual session would comment for the record on Oppenheimer's speech.But in fact it was warmly received, and at least a few bankers voiced the fervent wish - privately - that they could get Oppenheimer to speak at their annual company meetings while policy toward South Africa is questioned by shareholders.

In another sessiion on retail banking, Citibank Chairman Walter Wriston gave a gloomy account of the loss of consumer business by banks to other institutions, notably credit card companies and large retailers such as Sears, Roebuck.

He added that the smaller saver is being discriminated against because he can earn only 5 or 5 1/4 percent interest on savings "while the millionaire can get 7 to 8 percent. That unjust, but it will change over time," Wriston said.