Despite improvements in its program to match income and deduction information provided by payers with tax returns filed by individuals and businesses, the Internal Revenue Service failed to match 153 million items, or 22 percent of the data, it received in 1983, according to a report by the General Accounting Office.

The Information Returns Program (IRP), begun in 1974, is one of the major enforcement actions in the IRS campaign against underreporting of income by individuals. In 1983, more than $50 billion went unreported, IRS estimated.

Among the other highlights of the watchdog agency's report were the following:

*Although IRS computers identified 36 million cases in which taxpayers appeared to have underreported their incomes between 1979 and 1982, the service failed to follow up on 41 percent of them because of budgetary constraints.

*The IRS has been unable to develop a matching program for businesses, which have $500 million a year in interest and dividend income, because of differences in reporting periods and accounting methods.

*The IRS has been slow to impose penalties, as prescribed in 1983 tax legislation, on payers who send the IRS information in improper form. It declined to fine any of the 12,700 payers who were required to but did not file on magnetic tapes.

*The number of returns that are audited has dropped to 1.2 percent in recent years from 2.6 percent in the late 1970s, because of the complexity of the system and the lack of adequate personnel.

Testifying before the House subcommittee on commerce, consumer and monetary affairs, GAO's senior associate director, Johnny C. Finch, said IRS matching efforts were hampered by miscoded data, the cost of manually transcribing some paper returns and technical difficulties. Records containing 4 million pieces of information and amounting to more than $3 billion in interest and dividend income never were matched because 58 computer tapes were lost or erased. Moreover, he said, because IRS does not have advanced computers that can be updated continually, information that is filed late never makes it into the match system.

The GAO is Congress' auditing arm.

A pilot program to match business income and returns revealed that, in the majority of the 100 cases studied (53 percent), there was unreported income. It amounted to 10 percent of total income declared, but Finch said some of that might have been reported under the wrong heading on the return.

Although the IRS associate commissioner for operations, Philip E. Coates, agreed with much of the criticism in the GAO report -- which also gave the IRS credit for improvements -- he disagreed with a remark by subcommittee Chairman Rep. Doug Barnard (D-Ga.) that "billions of[tax] dollars are slipping through our fingers." He said the figure was in the millions of dollars.

Although the bank lobby succeeded three years ago in defeating automatic withholding of a percentage of interest and dividends, backup withholding for those individuals who do not provide proper identification has amounted to $151 million thus far, the GAO found.

In addition to flagging those who underreport their income, the IRP match has ferreted out 10.4 million taxpayers during the past four years who filed no return.

Coates said the IRS will be able to match 95 percent of the documents received this year. Because of problems matching business returns, the IRS will rely more on direct examinations of business records.