Federal workers, military personnel and retirees will take a multimillion-dollar drubbing when Senate and House conferees get through work on a compromise budget reconciliation bill.

Although the Senate and House budgets differ in their impact on government types, both plans would reduce the frequency of cost of living (COL) raises for U.S. retirees. Also to be considered by the conferees -- who will begin meeting either today or Monday -- are proposals to eliminate minimum Social Security benefits that go to 3 million people (including more than 200,000 retired feds), deliberalize benefits for federal workers injured on the job and make changes that could raise federal health insurance premiums as much as 100 percent next year.

The 100,000 federal and military retirees in metro Washington now get COL raises to keep up with inflation each March and September. Both the Senate and House have agreed to limit retirees to a single COL adjustment each year. That means the next raise will be in March 1982. Retirees who had counted on a COL adjustment (of at least 4 percent) next September will not get it then.

Conferees are also considering proposals that would limit persons eligible for Social Security benefits to the exact amount earned. Currently there is a minimum benefit of $97.50 per month for people under 65, and a $122 monthly minimum for persons 65 and older. The change conferees are considering would not take away any earned benefit, but it would eliminate the minimum benefit. The question is whether to make the change effective for everybody -- people now drawing the minimum and those who will get Social Security in the future -- or to grandfather in people already getting the minimum and apply the new rules to people becoming eligible for Social Security in the future.

The conferees will also decide whether to approve or kill Reagan administration proposals that would make drastic cuts in benefits for persons under the Federal Employees Compensation Act (FECA). Those changes, outlined here last week, got into the House Budget package although no hearings were held on them. Many people think the changes are fair and just, but the idea of slapping them into law without hearings troubles many members of Congress.

One change to be considered that will effect nearly everybody in government deals with the question of who is the primary payer of medical costs for federal retirees who have government health insurance and are also eligible for health coverage under Social Security.

Legislators who want to cut Social Security outlays would require that the Federal Employees Health Benefits program pay most medical bills of retirees that are now paid by Social Security.

The House version -- sure to be modified one way or another -- has conflicting language that could cut off all medical payments from FEHB and Social Security. If conferees decide that the FEHB should become the primary payer it will jack up premiums for almost all federal workers and retirees next year. One plan offered by Aetna, which has many retired federal workers in it, estimates its rates could go up 400 percent if the change is made.

It will be some time before the dust clears and people know what the conferees have decided. But for federal workers, none of the choices will be good.