The front of the building that houses the D.C. Office of Tax and Revenue. (Bill O'Leary/The Washington Post)

The D.C. government’s tax office erroneously withdrew $7 million from the bank accounts of hundreds of taxpayers because of a “computer error,” officials said.

The Office of Tax and Revenue said Thursday afternoon that the error occurred when it inadvertently processed income tax returns from other years, affecting 581 people who had previously authorized electronic bank ­withdrawals.

Officials said that they began contacting those affected shortly after the mix-up on Wednesday morning and that the money had been credited back to the accounts by Thursday morning.

The unintentional withdrawals are the latest embarrassment for an agency that has tried to make improvements after years of criticism amid high-profile improprieties and questions about how well it safeguards taxpayers’ financial information.

Bezalel Stern, a D.C. lawyer and resident, said he realized that $6,500 had been taken from his bank account Wednesday morning after his attempts to withdraw money from two ATMs were declined because the account had been emptied.

It took several calls to his bank before he found out that the D.C. government had cleaned out the account — leaving him with an overdraft fee.

“The money was basically stolen from me. You expect this from the bad guys,” he said. “You don’t expect this from your government.”

Stern said he called the tax office Wednesday morning after tracing the withdrawal through bank records. He said the amount withdrawn matched what he paid D.C. after filing his income taxes for the 2014 tax year.

Stern said an official at the Office of Tax and Revenue who took his call Wednesday told him that the error occurred when the office was testing new anti-fraud software. The official told him that the office was using last year’s tax filings during the test but that the “last step” — withdrawing money from filers’ accounts — was not supposed to happen.

“So, I basically paid my D.C. taxes a second time,” he said.

The official he spoke with told him that the money would be restored “over the next couple of days,” he said.

By Thursday morning, his account showed the transfer of the money back into his account was “pending,” he said.

The erroneous withdrawals came about two weeks after the office announced new procedures to combat tax fraud and identity theft. Two new computer­-based programs would help detect attempts at identity theft, officials said.

Natalie Wilson, a spokeswoman for the office, said in an email that the mix-up occurred when “a test file was erroneously placed in production.” Payments were credited back to the accounts within 24 hours after the mistake on Wednesday morning, she said.

The Office of Tax and Revenue said it would reimburse those who incurred fees as a result of the error. Those fees are expected to be minimal, Wilson said, and will come out of the office’s regular budget.

The agency has been at the center of controversy over the past decade.

In 2014, a former office employee admitted to using the financial details gleaned from her work to steal more than $114,000 from a retiree’s bank account. Two years earlier, another former employee was sentenced to 30 months in prison for stealing $400,000 in false tax credits using taxpayer information. And in 2007, investigators discovered a massive employee fraud conspiracy in the office that had drained more than $48 million from District coffers.