ILLEGAL POLLUTION by the federal government's nuclear weapons plants is intolerable. But the costs of remedying it are going to be extremely high, with the work running far into the next century. The estimates for the next several years are rising, and they will have a special impact because President Bush is going to have to fit them into his budgets.

For the current year, Congress appropriated $2.6 billion to begin bringing the weapons plants into conformity with the country's environmental laws. The Energy Department, which runs them, estimated late last year that through 1995 the work would cost about $4 billion a year. Now it suggests it may run as high as $6 billion.

The most dangerous of the violations involve toxic wastes that have been allowed to escape into groundwater or the atmosphere. Many of these cases began a long time ago, shielded from public view -- and from the enforcement routinely applied to private industry -- by the veil of secrecy that for four decades was wrapped tightly around these installations.

There are close similarities between the renovation of the weapons plants and that other monumentally expensive cleanup, the liquidation of the bankrupt S&Ls. Each could have been resolved at much lower cost if the government had chosen to act earlier. And both will push the budget deficit rapidly upward.

Now that Mr. Bush has conceded the urgent need for more taxes, a lot of Reaganites are busily at work weaving the myth that under them the deficit was coming down smartly and the present embarrassing reversal is entirely the fault of their successor. In fact, the Reagan administration chose to ignore the bills for the collapsing S&Ls and for the weapons plants' pollution. In the case of the weapons plants, the Energy and Defense Departments knew in the early 1980s that they were worn, obsolescent and, by the standards of the civilian nuclear industry, dangerously inadequate. But the Reagan administration was engaged in a military buildup and had no interest in embarking on long and extremely expensive renovations. It was only at the end of that administration that, with much prodding from Congress, the magnitude of the pollution became clear.

The weapons plants and the S&Ls are not the only reasons why this year's deficit will be sharply up from last year instead of down. But they are major contributors to the need for higher taxes. Mr. Bush is being forced to deal with the enormous liabilities that his predecessor ran up but never acknowledged.