THE "NOT ME" game is going full steam since Washington suddenly discovered that we have to slash the federal budget, and it has started to result in the usual nonsense.

You know how it goes: The best way to force efficiency on HEW is to give it fewer funds, while the best way to do the same for the Pentagon is to give it more money. (Great logic.) Or: If you reduce spending for my program now, "it will only cost more in the long run." (Sure.) Or simply: You can't take it out on my group. (Yes we can.)

Indeed, we can take it from almost every group, working on that ancient principle of economic, political and social equity: Misery loves company. This is eminently clear to me after 23 years of frustration over executive agency budgets, exasperation at Senate hearings and bewilderment over the extent of government waste documented in reports by the Congress and its General Accounting Office. It is also reinforced by an excellent study on "Reducing the Federal Budget" recently issued by the Congressional Budget Office.

From all this, it is evident that we can cut the budget in three broad ways. First, hit programs which are not justified under any circumstances; it is even time we began thinking about eliminating entirely one federal agency a year, which would not be as traumatic -- or even as noticeable -- as some might imagine. Second, there are the marginal programs that cannot be justified during this time of rapid inflation. Finally, there are the painful emergency adjustments that must be made to meet our current economic crisis.

While President Carter has moved in the right direction, his call for $13 billion in cuts for the next fiscal year does not go nearly as far as needed. It cannot conceivably erase a federal budget deficit now projected at a minimum of $20 billion and likely to rise to at least $25 billion. We must cut more deeply.

It is impossible to list all of the thousands of programs that could reasonable be trimmed. But while we await the president's specific proposals and the interest groups keep howling "not me," consider the following cuts in outlays -- forget the misleading "budgetary authority" numbers -- for the next fiscal year.

1. Unjustified Under Any Circumstances. Heading this list is revenue sharing. We are "sharing" (meaning giving) $6.8 billion in revenue, money. Washington does not have, to states with recent budget surpluses. The states end up spending money they don't have to raise, leading to federal dollar addiction. The withdrawal pains will be severe but necessary.

Three of my favorite indefensible government expenses go for travel, consulting and public affairs. We will spend next year almost $9 billion on travel, $2 billion for consultants and $1 billion to support 20,000 federal public information and communications specialists. We even hire consultants who travel to Washington to assist in public relations -- a remarkable combination.

The travel and consulting budgets should be reduced by $1 billion, as recommended by Democratic Sens. James Sasser of Tennessee and David Pryor of Arkansas, and public affairs could be cut by one-quarter, saving $250 million. Does the Pentagon really need 1,500 public affairs specialists? Does HEW need more than 400? Come on.

Our waterways, highways and airports obviously are essential to interstate commerce, but why shouldn't the users pay a realistic price? By insisting that user fees reflect true costs, airport and waterways revenues could displace $2 billion in federal spending next year. In addition, Congress should lower the highway trust fund by at least $1 billion, helping to conserve not only money but energy as well.

How many Americans would vote for a program with no sound economic justification, one that provides a high subsidy to a very few individuals, contradicts national agricultural policy and aggravates flood problems? Many of our water resources projects, dams, waterways, canals and reclamation projects do just that.

There are at least 52 questionable water projects, such as the Tennessee-Tombigbee system in Alabama and Mississippi, the Russell Dam in Georgia and South Carolina, the Red River Waterway in Louisiana, the Stonewall Jackson Dam in West Virginia, the Central Arizona project and the Bonnevile unit of the Utah project. We could save at least $250 million immediately by canceling them and paying termination costs. Similarly, channelization and erosion control programs could be reduced by $94 million. Channelization is particularly destructive to wildlife habitats and is based on obsolete water control concepts.

Some savings can be made with simple steps. Take the income-reporting system for public assistance. There is no common reporting system for estimating the monthly income figures used for federal payments under the Supplemental Security Income or Aid to Families with Dependent Children programs. With accurate statistics, we could save $325 million next fiscal year alone and $1.9 billion over five years. This would impose no burden on the needy, only weed out those who are ineligible for assistance.

In three areas the administration in the past has proposed substantial savings, but the Congress has failed to go along. This is the year Congress must change its mind about the Clinch River Breeder Reactor, impact aid for schools, and reform of the Blue Collar Wage Board pay system. If Congress again moves forward with the Clinch River Breeder, the president should request a recision of these funds, saving $220 million. By stopping Clinch River and paying a termination fee of $150 million, we could still save $1.4 billion over five years.

Impact aid to local school districts with high proportions of federal employes long has been controversial. Eliminating the $250 million annual payment would cause some local hardships, but it would reflect the fact that federal employes already contribute substantially to the local tax base.

Accepting the administration's plan for blue collar pay comparability with the private sector would save $260 million. If Congress refuses to take up this offer, the administration should reduce blue collar employment by a similar amount and do so every year until the changes are made.

Finally, there are two government agencies whose justifications for existence have simply run out. They should be at the top of the eliminate-an-agency-a-year list.

The Law Enforcement Assistance Administration has failed to help solve the nation's crime problem or to improve the criminal justice system. It has merely become a budgetary crutch for local law enforcement authorities. We don't really need a $2 million prototype James Bond police patrol car with digital terminals, head-up displays, periscope rearview mirrow, microcomputer mass memory, tire sensors and so on. That is what LEAA gave high priority to.

The Small Business Administration is a more difficult case. While serving some small businessmen well, it has been plagued with questionable loans and outright fraud. The small businessman has often found it easier to obtain credit from other institutions. Eliminating SBA and LEAA would slash spending by $1.4 billion.

2. Marginal Programs. We must also cut back programs which could be justified, in some measure, during times of surplus finding but which now must be set aside. That is where economy begins to hurt.

Start with the Comprehensive Employment and Training Program. By cutting selectively in public service employment, youth employment and private sector programs, CETA outlays could be reduced by $2 billion next fiscal year.

Education programs cannot be exempt. We could pare federal elementary and secondary programs by $300 million, vocational education by $100 million, basic opportunity grants for college students by $100 million, college loans by $200 million and other small programs by $100 million. Total saving: $800 million.

It is also time for retrenchment in health spending, which has increased dramatically in recent years. Various legislative changes, including Medicare and Medicaid reforms, would net $270 million; delaying the new child health program and improvements in Medicare and Medicaid for a year would save $560 million. We should also adopt the Congressional Budget Office option of charging for food product inspection, which would save $130 million. At the various administrative offices of our national health programs, savings of $450 million could be obtained by specific retrenchments.

But social programs cannot bear the brunt of fighting inflation. There can be no sacred cows in the budget -- and that includes defense spending.

The Pentagon should immediately freeze all civilian hiring. With 991,000 civilians working there, we can afford to cut back by attrition. A 6 percent attrition in man-years in civilian personnel would save $1.2 billion.

Nearly $36 billion in major defense contracts are awarded every year without competiton -- 73 percent of the total. Government and private studies indicate that competition would save 10 to 25 percent over "sole source" awards. The conservative 10 percent figure, with half the saving phased in next year, would net $1.8 billion.

Another large and questionable chuck of Pentagon spending goes for "Independent Research and Development." These little-noticed payments hand defense contractors more than $1.4 billion a year -- with little accountability or measurable payoff for the defense effort. They should be cut in half.

There are a number of other ways to reduce Pentagon spending, again without any harm to our military capability: Eliminate military servants ($10 million); mothball obsolete World War II Naval Reserve destroyers ($130 million); improve the officer-enlisted men ratio ($20 million); cancel the manned penetrating bomber studies ($50 million); delay the Advanced Attack Helicopter and SURTASS ($420 million); rescind procurement of the CVN71 nuclear aircraft carrier from last year ($60 million, with greater savings in succeeding years), and rescind the trouble-plagued CAPTOR mine program.

To this list I would add a more controversial item -- the MX missile. We are concentrating on a futile deployment system and encouraging a similar Soviet scheme that would be unverifiable and destructive of furture arms control options. We could upgrade our sea-based deterrent while continuing to exploit cruise missile technology in lieu of the MX, saving $870 million next year and tens of billions later.

Next, foreign aid. The polls have long indicated that this is the most unpopular federal program. We cannot be shortsighted in our relations abroad, but we must be frugal.

Next year's foreign aid spending could be cut by $1.5 billion by reducing the International Development Cooperation Agency $500 million; general administration by $64 million; international organizations and conferences by $48 million; military assistance by $150 million; the Export-Import Bank by $176 million and the multilateral development banks by $300 million. The last two cuts will create anguish among those who have received informal pledges of U.S. support, but they must realize that governments can and must change their minds when new economic problems arise.

Members of Congress often work to protect budgets of the committees they head, but since economy begins at home, I offer the following recommendations for agencies under my jurisdiction as chairman of the Senate appropriations subcommittee on HUD and Independent Agencies.

NASA has recently submitted a proposed budget cut of some $760 million to OMB. The administration should accept this and postpone new programs, cut back on overhead and stretch out the space shuttle. This would be one of the deepest cuts in any government agency.

With housing money, which is tied to past commitments, it is difficult but not impossible to make substantial inroads. A solid cut in staffing, community development grants, operating subsidies, Urban Development Action Grants, research, and comprehensive planning grants could squeeze out savings of $187 million. And if we held the National Science Foundation budget to last year's level, $100 million more would float free.

"Countercyclical aid" is a much improved form of fiscal assistance targeted to areas of greatest need. But the $1 billion in planned spending may not be necessary if the budget is balanced and some semblance of order returns to the economy. Furthermore, a program this modest probably has no significant effect on employment.

If Federal Workman's Compensation and Civil Service Disability eligibility requirements were tightened and abuses eliminated, we would save $49 million. And hardly any budget has grown so fast as that of the National Foundation for the Arts and Humanities. Austerity demands that spending for these cultural activities be limited to last year's level -- saving $30 million.

Congress itself cannot escape the budget scalpel. Legislative branch appropriations should be cut by freezing new hiring, eliminating frills from the Hart Senate Office Building and restricting growth at the Library of Congress and the GAO.

Wherever possible, committee staffs should be cut. I have returned $350,000 to the Treasury over the past three years form my personal staff and office budget, and my Banking Committee staff has been reduced 6 percent since last year. We get along fine, just as other staffs could. Every congressional office should take a 10 percent cut in office expenditures this year.

The same holds for the bloated White House staff, including those "on detail" from other agencies. If the president does not set an example, his budget-cutting plea will simply lose credibilitly.

3. Emergency adjustments. To this point, the cutting pains have sometimes been acute, but they are within tolerable bounds. Two more difficult decisions could be made if the psychological and economic pressures behind our inflation rate are to be curtailed. They are emergency measures -- not to be taken lightly or extended over too long a period -- but they are necessary.

First, a federal white-collar pay cap could be set at 7.5 percent. This would be arbitrary, stringent and basically unfair; federal workers alone cannot fight inflation. If the other measures recommended here were accepted, however, the pay cap would be logical and essential to holding down wage inflation. The savings would be $950 million.

Second, $800 million could be saved if dual annual cost-of-living payments for federal retirees were limited to single adjustments. This would put federal retirees on the same basis as Social Security recipients.

All of the cuts cited here cannot be added up to give an overall total while about $32 billion in reductions are included, some spill over from one agency to another and others preclude precise analysis of the total economic effect. What is clear, however, is that there is enough room to balance the budget even under adverse assumptions and still provide for a tax cut later in the year should that be necessary. A balanced budget would also have the side effect of limiting interest charges on the deficit by about $2.5 billion.

Yes, there will be pain, and all of us in Congress will be deluged with the "not me" howls from interest groups and constituents. My own state will be hurt, for some examples, by cuts in revenue sharing, education and health spending and CETA jobs and higher airport user charges, and I have already heard, loud and clear, from affected constituents. cBut we have to cut more -- and we cannot rely on revenue-raising taxes. Going the revenue route would only tell the nation that, yes, once more, the president and the Congress are not really serious, are trying to avoid the very discipline they ask of everybody else, are playing bookkeeping games with our lives.

Only public opinion can turn this government around. Only national outrage over mindless spending will dent the rhinoceros hide of indifference within the bureaucracy. Perhaps we are close to that point.