The Internal Revenue Service center here, where the District and Maryland send 2.2 million tax returns a year, is in a heap of trouble.

Computer failures and human errors have produced the slowest processing of returns in at least five years, and brought on widespread communications breakdowns between taxpayers and tax collectors. Investigators also are looking into reports that some overworked employes destroyed tax returns.

Weeks before the April 15 filing deadline, congressional offices in Pennsylvania, which also sends returns to the center, already were besieged with constituent complaints involving both refunds and disputes from earlier years.

Accountants and lawyers tell these stories:

* A woman was assessed $100,000 in taxes related to her husband's estate, plus penalties, although she had already paid the $1,500 in taxes she owed.

* A Pennsylvania Realtor sent the IRS copies of his canceled $8,000 check on 10 separate occasions but liens were still placed on his bank account and house. The IRS had incorrectly recorded one digit in his Social Security number.

* The Philadelphia center failed to record $297 million paid in withholding taxes by 27,000 mid-Atlantic companies. Five bank accounts had been seized and thousands of threatening notices mailed before the center acknowledged the error.

* An agent visited a Waldorf, Md., modular-home company and deanded payment of $12,000. After the owner explained that all the tax had been paid, the agent called his office and came back saying the firm owed $4,900. He later changed the amount two more times.

* A company still has not been credited for a $93,000 payment in 1983 and a $25,000 payment in 1981.

* A minister was assessed for failure to make estimated tax payments and, although he kept registered-mail receipts of those payments, still hasn't convinced the IRS.

"If you got a notice from the IRS in the past, you could believe it. Now you can't," said a woman who has worked at the center for more than 20 years and asked that her name not be used. "We're giving people refunds to which they're not entitled and assessing people for payments for which they're not liable. We're not current on anything."

The General Accounting Office and the IRS internal investigations unit is looking into those stories and other questionable practices at the Philadelphia center.

According to reports in The Philadelphia Inquirer and anonymous calls and visits to the office of Sen. John Heinz (R-Pa.), employes are so overworked and discouraged that they have resorted to shredding or hiding returns to meet processing quotas.

One anonymous caller to Heinz's office said 20,000 returns had been run through the paper shredder because they couldn't be processed fast enough. Another caller reported the shredding of 7,000 returns. The Inquirer reported last weekend that returns also had been abandoned in the women's restroom, stuffed into drawers and carted out the door.

Acting Philadelphia director Joseph H. Cloonan said that an internal investigation is under way, and taxpayers who feel they are being dunned mistakenly should use the service's special problem-resolution program.

IRS officials in Washington and Philadelphia say some problems were inevitable, given the magnitude of replacing outdated data-processing systems with one computer, which became operational in January.

Cloonan said the Philadelphia center is working to cut down a backlog and is catching up on 1984 returns.

"I don't know why Philadelphia has had more problems. I think there was a period of time where we suffered some equipment failures that were extensive. The difficulty is, for every day you lose, it takes three to catch up," said Cloonan, who was brought in from another center in February.

The delays are nationwide, but Philadelphia is one of the slowest centers.

By the end of March, the Philadelphia center had processed -- coded, recorded, verified and posted -- fewer than half the number of returns it completed during the same period in 1984. The center, which handles returns from the District, Maryland, Pennsylvania, Delaware, the territories and citizens abroad, has processed 42.1 percent of the 4.4 million returns it has received. The comparable figure last year was 79 percent.

Nationwide, 57.8 percent of returns have been processed, compared with 77.5 percent in 1984. Philadelphia is second-to-last of the 10 centers; the Brookhaven, N.Y., center has processed 41.6 percent.

The Memphis center, where returns from Virginia and five other states are sent, has processed 65.1 percent of the returns it has received, compared with 81.4 percent by this time last year.

Memphis center director Raymond D. Keenan said in a telephone interview that his center was the first one where the new computer system was introduced, giving officials there more time to work through the bugs.

Keenan said the center "absolutely" will have no trouble sending all refunds to taxpayers by the June 1 deadline, after which the IRS must pay interest. The current lag between the time a return is sent and the refund mailed is about eight weeks.

Whether Philadelphia will meet that June 1 deadline is less certain. Officials say they are "on target," but the center is taking 10 weeks after a Form 1040 return is received to mail the refund. There are seven weeks between April 15 and June 1. The center is taking three weeks longer for refunds than it did last year and twice as long as in 1982.

Behind the delays is a new $103 million Sperry Univac 1100/84 computer system, which replaced older equipment at all 10 centers. For the new hardware, the IRS had to rewrite 1,500 programs. That was 3.5 million lines of computer programming, all of it in a new, more sophisticated computer language.

The final part of the new system was installed in January, just before the first returns came in.

"If I'd had my druthers, I'd have raised my hand and said, let's wait six months," said M. Eddie Heironimus, associate IRS commissioner for data processing. He said, however, that no matter when the computer was installed, the high-volume period would have been its most severe test.

Human errors have been at least as important a factor in the troubles in Philadelphia. Employes say employment practices in recent years have led to more mistakes, poor communications with the public and bad relations with management.

At the time of the foul-up involving failure to record withholding taxes for thousands of companies, center officials put off action for months even though they knew the computer tape was faulty and had not recorded the $297 million in payments. Workers in the computer branch did not tell the accounting branch about the technical problem.

In many smaller-scale cases, taxpayers say they can't get through. Letters go unanswered, payments unrecorded and telephone calls ignored, they say.

"I don't think IRS reads their mail," said Ronald L. Noll, an accountant from nearby Malverne, Pa. "Nothing you send them does any good. You can't stop that computer. There's no way you can get them to pay attention unless you go to a congressman."

The problems of his clients used to be resolved in days or weeks, Noll said. Now it takes months or years. The Realtor who sent in copies of his canceled check 10 times tried for three years to convince the service. And even then, he succeeded only by going through the office of Rep. Richard T. Schulze (R-Pa.).

Writing on behalf of the Philadelphia bar, attorney Mervin M. Wilf told IRS Commissioner Roscoe L. Egger Jr.: "There is reason to believe that many of these errors may never be called to the service's attention, because the taxpayers chose to pay the erroneous interest and penalty rather than incur the expense and bother of confronting the service. Worse yet, some taxpayers may have paid the amounts demanded in such notices out of fear and ignorance."

Schulze's office has received a higher number of requests for assistance from taxpayers than in the past. In Heinz's office, taxpayer assistance requests usually make up about 50 percent of the total caseload. This year, tax cases are about 70 percent.

Employes at the service center also have been calling Heinz's office to protest working conditions. For the last few years, most of the 2,800 workers at the center have been required to meet what they call quotas and Cloonan calls production standards.

For instance, a worker handling Form 1040s having simple itemized deductions is expected to enter 35.5 returns into the computer per hour -- less than two minutes per return. Cloonan -- who acknowledges that morale is "hurting" -- said that the standards are only an average, that employes have a say in setting them, and that they can be altered.