A House Banking subcommittee delivered a sharp rebuke to the Reagan administration policy on Africa yesterday by voting to bar U.S. support in the International Monetary Fund for any loans to South Africa because of its policy of racial segregation, or apartheid.

The vote brought to a head a long dispute between congressional critics of apartheid and the administration's "constructive engagement" policy of maintaining good relations with South Africa to influence its behavior.

Rep. Stan Lundine (D-N.Y.) said he came to support the anti-apartheid amendment "somewhat reluctantly." But he added, "We tried to resolve the loan issue with quiet diplomacy and were not even accorded a meeting with the adminstration . It seems we have no recourse . . . there being no other alternative to express our concerns."

The vote by the subcommittee on international trade, investment and monetary policy came during consideration of a bill that would increase U.S. contributions to the fund by $8.4 billion. The panel also approved that bill on a voice vote and sent it to the Banking Committee for consideration Monday.

The vote by the key subcommittee follows a controversial $1.1 billion IMF loan to South Africa in November to help it meet balance-of-payments deficits caused by last year's drop in gold prices. That loan was criticized by 60 congressmen, including the Congressional Black Caucus, in a letter to the administration last fall, and the United Nations General Assembly overwhelmingly voted to ask the IMF to reject the loan.

But the Reagan administration supported the loan in the IMF deliberations, saying that it was granted on purely economic terms. Administration officials argued that to consider the issue of apartheid during the loan negotiations would politicize the IMF.

Subcommittee Chairman Stephen L. Neal (D-N.C.), while criticizing apartheid, sought to persuade the panel that the political issue could come back to haunt it. "The IMF was designed to solve one problem in the world and that is to keep and open free trading situations," he said. He said it had done the job in an "outstanding way" by avoiding political tangles. In adopting the anti-apartheid amendment, he said, "we set a precedent" that could later be used against other countries that do not follow U.S. policies or norms.

Rep. Mike Lowry (D-Wash.), one of the sponsors of the anti-apartheid amendment, rejected opponents' charges that the amendment would politicize the IMF. He criticized the administration for not taking into consideration congressional opposition to the loan last fall and said it "is a question of what do elected representatives in this country stand for."

Echoing the concern of many of the subcommittee members about whether a harmful precedent might be set by the amendment, Lundine said it "causes me grave difficulty." But he said the IMF funds bill, which is "crucially necessary" for international stability, needed bipartisan support to pass through Congress and it faced problems in the House without the amendment.

The amendment passed on a voice vote after a substitute to weaken it was defeated, 8 to 9.

Much of the congressional concern centered on South Africa's need to borrow from the IMF. Opponents of the loan argued that because of its apartheid policies, South Africa has restricted its labor market and thus its economic policies. They also pointed out that although gold was $315 an ounce when the loan was requested, it had risen to more than $400 by the time the loan was approved and has stayed above $400.

These economic concerns surfaced during the IMF considerations of the loan in November, according to draft minutes that were made available to The Washington Post. Deliberations by the IMF executive board are made privately by the directors designated by the member nations, but the final decisions are usually announced.

Many of the representatives of Third World countries argued against granting the loan to South Africa, and all the opposition was couched in economic terms, according to the minutes. One of the key opponents was Saudi Arabia's Yusuf A. Nimatallah. He criticized South Africa for failing to achieve "genuine fiscal adjustment" to reduce the budget deficit and said it could borrow from the private sector since its credit rating was favorable and thus was "taking advantage of the fund's cheaper resources." His criticism is considered important because the Saudis are major contributors to the IMF.

Other countries that opposed the loan, according to the minutes, included India, China, Iran, Kuwait and most Arab member countries. The loan was supported by the United States, Britain, West Germany and France. Most of the African nations did not support or reject the loan, which passed with about 52 percent of the vote. The United States, because of its major funding, controls close to 20 percent of the vote.

The administration objected to the anti-apartheid amendment passed yesterday. Assistant Treasury Secretary Marc Leland told the subcommittee it would set a bad precedent. Richard D. Erb, the U.S. director at the IMF, testified that "the United States has not cast a vote on political grounds" in the fund and would object to that.

But Rep. Julian Dixon (D-Calif.), the leader of the Black Caucus and of the movement to bar U.S. approval of loans to South Africa, said in an interview yesterday that the Reagan administration has voted against Nicaragua in the World Bank and worked to reduce IMF support of Grenada for political reasons. Both are leftist countries.