Much to the annoyance of Treasury Secretary James A. Baker III, his highly trumpeted plan to defuse the Third World debt bomb has been greeted with increasing skepticism here and abroad in the absence of concrete moves to implement it.

Most recently, World Bank President A. W. (Tom) Clausen -- who resigned his post as of mid-1986 after the United States was unwilling to back him for another term -- took an unexpected potshot at the Baker proposal. In an interview with John M. Broder of the Los Angeles Times, Clausen termed the Baker plan vague and under funded.

What Baker had suggested in a speech last October in Seoul was that the World Bank, the Inter-American Development Bank and similar institutions double their projected loans to 15 key debtor countries over the next three years from $4.5 billion to $9 billion, and that commercial banks add an extra $20 billion to their loans over the same period.

"If you look at it, where's the plan?" Clausen asked. "Intellectually, it is sound as a concept, which is all it is. All the pieces are not yet clear. We've got to find a way to enlarge the flow of funds to the developing countries."

Clausen, who has been criticized by the Reagan administration as slow to deal with the debt crisis, told the Times that he had been disappointed by the kind of support he had gotten from the Reagan administration over the past five years. He is also known to believe that the bank had shifted gears to cope better with Third World debtors before Baker did so with his now famous "initiative" in Seoul.

But the importance of the Baker initiative was the recognition by the U.S. government that the debt crisis would not be solved by austerity alone: What the debtors urgently needed was the prospect for renewed economic growth -- a growth that would provide them some hope, along with the ability to service their debts.

Experts on Latin America agree that, with the stunning recent drop of oil prices, the situation demands even closer attention than before. In Nigeria (which depends on oil for 95 percent of its export earnings) and in Mexico (almost as dependent on its oil resources), there naturally is deep gloom over the crash in oil prices. The overwhelming fact of life is that there is more oil available in today's world than the demand to consume it -- and nothing is going to change that fact for many years ahead.

Thus, what benefits the United States, Japan, Western Europe and large Third World oil consumers such as Brazil and Korea at the same time will deepen the Latin American debt crisis elsewhere. Yet, as Clausen suggested, nothing much has happened since the Baker speech.

As might be expected, there is a much different view at the Treasury Department. In an interview, a senior official insisted that the plan is on track -- although perhaps too many expectations had been raised about early successes.

"I think you all are always looking for instant gratification," he said. "And I would think it's unfair because, number one, we didn't get into this mess overnight. And we're not going to get over it overnight. Secondly, we're talking about long-term macroeconomic changes in economic policy by these countries. Further, we got caught up in this exercise about who's going to be first.

"To some degree, we were at fault for answering questions that you all put to us -- who's going to be first, that kind of thing."

Actually, he said, Ecuador, Colombia, The Ivory Coast, Morocco and Uruguay already are participating in the kind of expanded World Bank lending that Baker visualized. But which country will be the "first" to qualify for additional commercial bank loans by reforming its economic system, as Baker demanded? The senior official says the problem is that "these countries cannot say they will be a guinea pig if it looks too much like kowtowing to Uncle Sam."

"BAKER PLAN, NO!" is the graffiti scrawled over walls in Ecuador, President Febres Cordero reminded Reagan administration officials on a recent visit here. This suggests how difficult it is for political leaders to accept the kind of economic discipline demanded as the price for new loans.

Yet, if critics contain their impatience, the senior official said, many debtor nations will utilize the Baker initiative, meeting the reform conditions without saying very much about it "because . . . there is no other alternative save . . . writing off the debt, which I think is a declaration of defeat."

The situation actually might be "a blessing in disguise," Fed Chairman Paul A. Volcker said publicly the other day. What Volcker means is that after 40 years of Third World nationalization, protectionism, discouragement of foreign investment and expansion of the governmental role at the expense of the private sector, the trend now may be reversible.

If Volcker is right, there is a rethinking process going on: It is becoming respectable in Latin America to talk about such things as privatization and other elements of an economic program that Baker might lump as "supply-side" concessions. If so, some new deals will be struck with the banks, and more loans will go forward under the Baker initiative. Whether they will be enough -- and in time -- to ease the tensions and overcome the political obstacles remains to be seen.