THE ADMINISTRATION, hardly started on its campaign to reduce the size of the non-defense bureaucracy, is already finding it prudent to reverse some of its earlier decisions. It turns out, upon closer inspection, that some bureaucrats do useful things--such as save taxpayers money.

Take, for example, the agents who audit returns and collect debts for the Internal Revenue Service. Last week Treasury Secretary Donald Regan said he needs several thousand more of them to crack down on tax evaders. The secretary estimates that each dollar spent on agents will increase tax collections by at least four dollars, thus reducing the burden that now falls on honest taxpayers.

The administration did another quick turnaround with respect to RIFs--that's "reductions in force"--in the U.S. Employment Service. Local employment service offices provide free job-finding help to unemployed workers--primarily the relatively unskilled-- and enforce the requirement that people receiving unemployment benefits search for new jobs. It can't compete with high-powered private agencies in finding jobs for high-salaried workers--companies and individuals are quite willing to pay the stiff fees that private agencies command. The employment service, however, provides the sort of low-cost, large-scale job- finding service that suits the needs of many workers and employers. In 1980, it received almost 7.5 million job orders and filled over half of them.

Last year, the administration sought budget cuts that would require laying off almost half of local employment service staffs. Congress agreed to the final cuts in the last-minute appropriation rush before Christmas--despite the concern of many members that it was especially unwise to reduce the chronically understaffed service at a time when unemployment is rising rapidly. Scarcely a month later, with many offices already closed and unemployment lines growing, the administration asked Congress to restore most of the employment service's lost funds.

It will be surprising if there are not many other examples of false economies in the administration's rush to RIF. Early on the administration set itself a goal of cutting the federal non-defense work force by 75,000. No justification has ever been produced for the choice of that particular number, nor has any plan been forthcoming that relates the distribution of reductions to actual overstaffing--or understaffing--of particular government functions. Laying off government workers can cost a great deal, not just in personal hardship for the workers involved, but also in lost productivity and services for the public.