It used to be that in the modern, environmentally controlled warehouses of Brazil, the excess supplies of green coffee beans would build up millions of gunny sacks each filled with 132 pounds. At one point there was enough excess to keep the whole world in coffee for a year.

No more. Today, almost two years after a devastating freeze, destroyed or damage half of Brazil's 2.9 billion coffee trees, the filled sacks are coming out of storage. They are the only thing preventing an actual shortage of coffee.

As a result, the world's coffee bean reserve, the wellspring to be tapped in times of shortage, is down to about four month's supply - from 16 months in 1966. The u.S.Agriculture Department estimates that the surplus will dwindle to 19 million bags - only three months of filled coffee cups - by the time Brazil recovers from the 1975 freeze two years from now.

That is if there is not another, freeze meantime. Shoulds there be another disaster, the world's coffee drinkers will experience an actual shortage.

The dwindling surpluses today and imaginable shortage later are the propellant of today's rocketing coffee prices. In the global auction-house of coffee, a souffle-likt market reacts to the slightest draft, and prices respond.

They respond to such degree that the average American family's coffee bill rose from $40 in 1975 to $60 last year, and unroasted beans that sold for an average $2.45 a pound on the world market in February sold for $3 one day last week.

A one pound can of Maxwell House coffee now sells for $3.89 and $3.39 at two Washington-area supermarket chains.

"It is the economic law of supply and demand," says Richard A. Smith, deputy administrator of the Agriculture Department's Foreign Agricul-demand remains the same, there is a ture Service. "When supply drops and reaction throughout the market."

But a debate has been joined between coffee, agriculture and market experts, on one hand, and some congressional and consumer representatives, on the other, over whether coffee prices have been manipulated beyond market forces to gouge consumers.

"I maintain that recent developments both inside Brazil and between Brazil and other countries have caused a distortion of (the) market and are the reasons coffee prices are so high," Rep. Frank Annunzio (D-Ill) told a congressional subcommittee last month.

"Sure, the frost is significant . . . (but) by withholding coffee from the market and triggering a doubling tripling and finally a quadrupling of the price, the Brazilians might just be hoping that the American consumer won't mind paying off some of Brazil's $28 billion national debt, Annunzio said.

"There is no evidence that Brazil, as a country, has a policy of holding back stock," said Julius L. Katz, assistant secretary of state for economic and business affairs. Katz said, however, that individual farmers and exporters in Brazil may temporaily hold their supplies off the market in the hope of getting a higher price, as American farmers sometimes have done with grain. They also risk getting a lower price, should the marketplace change.

The Foreign Agriculture Service estimates that Brazil will export just 500,000 fewer bags of coffee this year than last - 14 million vs. 13.5 million - despite severely reduced production. Smith also notes that Brazil has embarked on a $1 billion replanting program to restore coffee production to normal, which will eventually lower prices.

Katz says consumers have probably helped raise coffee prices somewhat: By buying and stockpiling coffee in fear of higher prices, they have up prices and indicating to the world's sellers that consumers will pay higher prices.

He cited a study concluding that stockpiling accounts for the equivalent of 2 million to 3 million bags of unroasted beans.

Despite higher prices, the world's coffee-consuming nation's are importing more coffee this year than in any of the previous three years, according to the agriculture service - 58.8 million bags now compared to 56.1 million the year before the 1975 freeze.

Americans sent $1.56 billion overseas for coffee in 1975; last year it was $2.72 billion, according to the Treasury Department, and this year it will be still more. Naturally, this has been a windfall for exporting nations. And, just as naturally, this has led to the charges of manipulation of the market by Brazil - with State Department complicity.

Reps Benjamin Rosenthal and Frederick Richmond, both New York Democrats, have released State Department documents that, they say, support their contentions of manipulation in Brazil and a policy of higher coffee prices as foreign aid.

Katz and other State officials say the documents subtantiate neither conclusion.

The congressmen's target is the International Coffee Agreement, a treaty between the world's coffee-consuming and producing nations. As negotiated in 1962 and again in 1968, the agreement set minimum prices and gave each exporter a quota, guranteeing the exporter a market and the importer a supply.

Citing a newly declassified State Department analysis of the agreement, Rosenthal charged that State itself views it as "an instrument of foreign policy and not as a mechanism to aid the American consumer." Rosenthal cited this portion of the analysis: "The first agreement and, to a lesser degree, the 1968 agreement were both signed in periods of low coffee prices and surplus production. They were presented as instruments of our foreign policy complementary to the Alliance for Progress."

A State expert on coffee confirmed that was true of the 1962 and 1968 agreements but not of the current one, which was negotiated in 1975. He cites another part of the same study that quotes presidential instructions to negotiators to seek "more substantial protection of consumer interests," particularly protection from rising prices. That was not negotiated, but State feels it did not get other consumer benefits in the treaty.

The quotas and minimum-price provisions are irrelevant to current coffee prices: They have not been in effect since 1972 and will not take effect again until world prices drop to a range of 63.5 to 77.5 cents a pound.

By almost all accounts, prices will not start heading in that direction until 1979, barring diaster. So, unless consumption declines the prospect is rising prices.