The Internal Revenue Service has been so lax regulating the special retirement system for railroad workers that railroads paid the pension fund about $73 million less than they should have between 1986 and 1989, the chairman of the House Committee on Government Operations charged yesterday.

Rep. John Conyers Jr. (D-Mich.) wrote Office of Management and Budget Director Richard G. Darman that an audit of the federal Railroad Retirement System also showed that the Treasury Department last year had failed to transfer to the retirement trust fund $156.6 million in retirement taxes paid by the railroads. The Treasury sent the money to the trust fund after a query from the inspector general of the Railroad Retirement Board, which administers the pension program for railroad workers.

Railroads make payroll tax payments to the fund much the way other private employers pay Social Security taxes. The underpayments to the fund come at a time when Congress is again being asked to bail out railroad retirement.

In 1983, Congress passed the Railroad Retirement Insolvency Act because the fund was financially strapped. Another bailout bill is pending in the Senate.

The fund is in potential trouble because employment in the railroad industry is shrinking, leaving fewer workers to support retirees in the future. The government has estimated that in a worst-case scenario, taxpayer liabilities could hit $30 billion. The trust fund now has $9 billion and covers about 300,000 workers.

"The inspector general's audits clearly demonstrate that the insurance and retirement funds of American railroad workers and their families are well below the radar of protection against financial predators," Conyers said in his letter to Darman, whose agency has been asked to testify on the issue next month because it oversees the federal budget. "Railroad employers' payroll taxes are not scrutinized by the Internal Revenue Service, internal auditors or public accountants."

The Office of Management and Budget has also been looking into the railroad retirement system. A high-ranking OMB official yesterday described the Railroad Retirement Board as a "management morass." He said a recent investigation by the OMB found the retirement board to be "riddled with problems" and he too praised the work of the board's inspector general. He said it was "absolutely true" that oversight of the retirement system had been as lax as Conyers charged.

The Government Operations Committee will hold hearings on the lack of oversight for the railroad pension system as part of what Conyers said is his concern over the "hollowing out of government" as a result of the budget cuts during the Reagan administraion.

The Association of American Railroads, the trade asociation that represents the industry, referred calls to the retirement board.

As an example of the abuses, auditors cited an unidentified railroad that did not pay taxes from fiscal 1986 through 1988 on 7.8 million hours that employees worked.