The International Monetary Fund is still bitterly condemned in parts of Latin America as ''neo-colonialist'' or ''neo-imperialist'' for its role in enforcing austerity as it managed the 1982 debt crisis.

But things may be taking a turn for the better under the agency's new managing director, Michel Camdessus. He is highly sensitive to the fact that the IMF is a bad word in the Third World, a convenient whipping boy for politicians who duck blame for their own questionable decisions.

Camdessus has faced the reality not only that the IMF's image needs repair but that the IMF must take into account the ''social impact'' of its programs. This is a remarkable departure for an agency that until now has refused to acknowledge its programs had any effect on income distribution.

A resurrection of the reputation and clout of the IMF, if that develops, could not come at a more propitious time. New leadership on the international economic front is desperately needed: the World Bank, under President Barber Conable, is likely to be mired a while longer in internal problems stemming from its unhappy reorganization experience. Paul Volcker, who worked effectively in the international debt arena, has now departed his post on the Federal Reserve Board.

And few Washington observers expect the Reagan administration, weakened by the Iran-contra scandal, to take any major initiatives relating to the debt crisis. So it may fall to Camdessus and Conable together to assume management of the debt problem.

Conable has big problems at the World Bank. Starting at next month's annual meeting of the two organizations here, he'll have to try to purge the feeling that he's blundered on the reorganization issue.

Camdessus, on the other hand, has already made a strong impression. He's open and accessible to staff. ''He even recognizes that arrears -- overdue payments to the IMF by some countries -- are a problem, and that's something that the IMF didn't even want to talk about before Camdessus arrived,'' says a close observer.

In his first major initiative, the former French national bank governor is trying to put together a special fund to help hard-pressed African countries. ''Here's the IMF, which is supposed to be a short-term monetary institution, concerned about international liquidity and exchange rates,'' says expert Richard Feinberg of the Overseas Development Council. ''Yet, this man Camdessus is traveling around the world looking for soft money for Africa. That's pretty surprising.''

Of course, Camdessus is motivated by more than humanitarian concerns: the IMF is heavily exposed in Africa with loans to nations that can't be paid off. He put his proposal -- for tripling concessional aid funds available for Africa and other low-income countries -- to the heads of government at the economic summit in Venice and got a good reception. Later he called for stretching out debt repayments for this group of countries, and for a program of ''lasting adjustment'' worked out by each country with the IMF and the World Bank.

Reagan administration officials are still counting on the bank to play an increasingly important role in stimulating economic growth in Latin America. In fact, Treasury Secretary James A. Baker III looks to the bank to replace, in substantial part, help that otherwise would have flowed through the Inter-American Development Bank, before an unhappy policy deadlock soured U.S.-IADB relations.

Given all of the notoriety surrounding the World Bank's reorganization, Conable's speech to the joint annual meeting could be a make-or-break event. Bank staff are burning the midnight oil trying to come up with new initiatives and ideas that can sound credible.

There is hope at the bank that Conable will address directly the serious question of resource transfer, which has been downgraded in the past by bank officials such as Senior Vice President Ernest Stern. Beginning in 1982, the Third World went from being a recipient of funds to a being a major transfer agent of funds to the North. That would seem to be contrary to the bank's major mission. To help reverse that process, Conable needs to force action on a general capital increase.

Feinberg thinks that Conable should also define how the bank is going to deal with the poverty issue, after some years of giving it a less-than-urgent, trickle-down priority. ''I think he's got to allay fears that the bank has forgotten about poverty,'' Feinberg said.

Camdessus observed the other day that while the current debt strategy needs ''fresh impetus . . . a resumption of normal debtor-creditor relations is . . . not yet imminent, to say the least. Indeed the debt situation has recently been showing some signs of strain. All this suggests that it will take longer to resolve the debt problem than had been hoped in the light of progress made in 1982-84.''

If Camdessus can restore the IMF's image, if Conable can restart the World Bank and if the agencies can actually work together, there may be some long-run hope.