Retiring Comptroller General Elmer B. Staats told Congress yesterday that the Reagan administration has ignored billions of dollars worth of potential budget savings, including what he called "absolutely essential" curbs on automatic cost-of-living increases in Social Security and other federal benefit programs.

While agreeing with many of President Reagan's proposed belt-tightening measures, Staats said in his farewell testimony before the House Budget Committee that the President's package of more than $40 billion in budget costs went too far in some areas and not far enough in others.

Warning that there is a "long, long way" from proposing a spending cut and achieving it, Staats also cautioned Congress against cutting taxes before it nails down its budget cuts, especially in light of what he described as uncertainties over the administration's assumptions about inflation and economic growth.

Staats drew some favorable nods from committee members in proposing that the President and Congress be given substantial flexibility in modifying the current system under which programs accounting for 30 percent of all federal outlays are increased automatically to conform to rises in the Consumer Price Index (CPI).

The so-called CPI indexing of Social Security, food stamp, federal pension, child nutrition, black lung and other federal programs accounted for an increase of nearly $30 billion in the 1980 budget, nearly $20 billion for Social Security alone, according to the General Accounting Office. In future years, the figures are expected to rise dramatically.

Both House Budget Committee Chairman James R. Jones (D-Okla.) and Senate Budget Committee Chairman (Pete V. Domenici (R-N.M.) favor curbs on indexing and Senate Majority Leader Howard H. Baker Jr. (R-Tenn.) yesterday endorsed the idea as "sound" in principle.

But many lawmakers flinch from the political repercussions of tampering with Social Security, and David A. Stockman, Reagan's budget director, told the Senate Budget Committee last week that it would be "wrong as a matter of social policy and incorrect as a matter of economic policy" to make beneficiaries of federal programs suffer because of inflation. A leading Democrat on the House Budget Committee said yesterday it would be politically difficult to change the indexing system without administration backing.

In steering Congress toward savings overlooked by Reagan, Staats also said more than $10 billion could be saved by more vigorous debt collection and pursuit of misused funds. Closing of tax loopholes and program changes such as elimination of Davis-Bacon rules setting high wage rates for federal construction jobs could also save billions of dollars, he said.