The White House budget office says it has found a $650 million special-interest giveaway to the railroad industry in the Senate Finance Committee tax bill drafted to carry out the deficit reduction agreement between the administration and congressional leaders.

Budget Director James C. Miller III has protested that the little-noticed provision, sponsored by Sen. Max Baucus (D-Mont.), undercuts committee claims that the tax bill had been kept "clean" and free from such tax breaks.

Miller's complaint, made in a letter to Finance Committee Chairman Lloyd Bentsen dated Dec. 3, drew an angry denial from the Texas Democrat. Bentsen wrote that he was "shocked" by Miller's letter, adding that it "can only serve to undermine the budget agreement" between President Reagan and congressional leaders.

Bentsen and other senators on the panel are sensitive to charges that their bill contains narrow-interest tax breaks because they drafted the legislation under pressure to avoid any provision that would drain revenue from the Treasury.

The administration has insisted that the tax hikes envisioned under the deficit-reduction accord, which total $9 billion in fiscal 1988 and $14 billion in fiscal 1989, must not include revenue-losing items of the sort that usually accompany tax bills. Bentsen's House counterpart, Ways and Means Committee Chairman Dan Rostenkowski (D-Ill.), has grudgingly agreed to abide by that rule provided the Senate panel does the same.

When the Finance Committee approved a tax-increase package last Thursday, Bentsen said it adhered to the terms of the budget pact. Treasury Secretary James A. Baker III agreed. Treasury officials aren't supporting the protest by Miller, who said in his letter that the railroad provision, while not costing revenue in the short run, would do so in the long run.

The railroad provision would allow taxes paid by railroad retirees on their pension income to continue flowing into the federally administered rail pension fund instead of into general government revenues. The provision, according to Miller, would bestow preferential treatment on the rail industry pension system. "The rail sector, not American taxpayers, should pay for their rail pensions," Miller wrote.

In his 1984 race for the Senate, Baucus received more than $17,000 in campaign contributions from political action committees sponsored by railroads and rail unions, according to Federal Election Commission records. But a Baucus spokesman, Scott Williams, discounted the importance of the campaign contributions, asserting that a more significant factor is the nature of Baucus's constituency.

"Montana is a rail state," Williams said, noting that two major rail lines run through the state and that about 12,000 railroad workers and pensioners live there.

According to Williams, the tax-bill provision under attack by Miller is needed to shore up the troubled finances of the rail pension fund while a commission considers ways to overhaul the system. "We want to keep things going and find a solution," Williams said.

The rail pension system is a government-run hybrid, existing in a quasi-public netherworld between Social Security and private pensions. Established by Congress in 1937, it was originally designed to be supported solely by railroads and rail employes. Later, however, it became increasingly dependent on financing from the Treasury and from the Social Security trust fund, partly because the number of retirees far surpassed the number of active workers in the industry.

One way in which Congress sought to help the system was by allowing railroad retirees to receive most of their pension benefits tax-free -- unlike private pensioners, who must pay taxes on their benefits once they have recovered their contributions. In 1983, Congress ended the tax exemption. However, lawmakers -- concerned about the rail program's persistent financial problems -- approved a five-year transition during which the taxes collected on rail pension payments would flow back into the system.

Baucus's provision would extend this transition period until 1990. In his letter, Miller admitted that the Baucus proposal wouldn't immediately affect federal revenue, because it would simply continue to divert money from one government coffer -- the Treasury -- into another, the railroad retirement trust fund.

But the action would lose revenue in the long run, Miller wrote, because "at some point, the rail sector will be able to substitute this subsidy for their own contributions to pay pension benefits. The federal government will then be short those receipts that the rail sector would otherwise have to contribute to pay these benefits."

The amount of money that will be diverted from the Treasury into the railroad retirement fund would be $125 million in fiscal 1988, $257 million in fiscal 1989 and $268 million in fiscal 1990, according to administration projections.

In his response, Bentsen accused Miller's agency of harboring "a longstanding hostility to the railroad retirement program."

The lawmaker also disputed Miller's assertion that the Baucus proposal would lose revenue. "The 'revenue loss' you speak of is a completely hypothetical one based on speculation that this provision will at some unspecified future date influence Congress to enact or not enact changes in the railroad retirement tax rates," Bentsen wrote.