The fraudsters come from all walks of life — barbers, postal workers, check cashers and even police — all tempted by a criminal enterprise some consider more lucrative than selling drugs. Their segmented but sophisticated networks swipe billions of dollars from the U.S. government each year even as authorities increase their efforts to stop them.

Their scheme, known as stolen identity refund fraud, is simple: steal unsuspecting victims’ personal information, then use it to direct tax refunds to themselves. And in the past three years, authorities say, their misdeeds have increased exponentially, prompting the Internal Revenue Service to launch nearly 1,500 investigations in the past fiscal year, up from just 276 in fiscal 2011.

“This is something that is becoming absolutely rampant throughout the country,” said IRS Special Agent Joy Cuffee, a spokeswoman for the agency’s criminal investigators.

In one recent case in the District, the affable owner of a barbershop in Southeast Washington admitted to leading an organization that tried to claim more than $20 million in fake tax refunds, often using the identities of drug addicts, incarcerated prisoners and those in nursing homes. In Baltimore, a police officer admitted to searching law enforcement databases to find information that might be used on tax returns.

And in federal court in Alexandria last week , a former Dominican soldier and barber was sentenced to five years in prison for trying to bring a fake tax refund operation from New York to Northern Virginia.

The man, Leurys Antonio Olivo, asked forgiveness of Americans at his sentencing hearing, noting that “it was the citizens of this country that were the victims of my actions.” Prosecutors say that the identities of about 60 people were stolen in the scheme and that the IRS issued about $352,000 in refunds that it should not have.

The case is just a small example of the broad financial implications the crime has for U.S. taxpayers and the personal headaches it imposes on those whose identities are stolen.

A recent report from the office of the Treasury inspector general for tax administration found that $3.6 billion in potentially fraudulent tax refunds was issued in tax year 2011. The IRS says a typical case for a taxpayer whose identity is stolen takes about 180 days to resolve. That is an improvement from another Treasury inspector general’s report that found an average resolution time of 312 days for the sample of cases it reviewed from fiscal 2012.

Last month, federal authorities conducted community meetings and issued tips on avoiding identity theft, though they acknowledge the problem is sometimes unavoidable, fueled by corruption among those preparing tax returns, cashing checks or handling people’s personal information.

Greg Hunter, Olivo’s attorney, said the criminal enterprises are loose collections of people out to make money rather than true networks. The endeavor, he said, can be more lucrative than selling drugs.

“In the tax refund business, you can start with no money, you can fill out free tax forms,” Hunter said. “If it succeeds, you get a check, and if you cash the check, all that money goes into your pocket. That never happens to people selling drugs.”

Prosecutors say Olivo, who lived in the Bronx, was hoping to expand his fraud business to Northern Virginia, and he recruited Juan Alexis Lima-Castillo of Fredericksburg to help him cash checks issued by the IRS. Lima-Castillo, who briefly served as an informant for investigators, was sentenced to 60 days in prison for his role in the scheme.

His attorney, Steven L. Duckett Jr., wrote in court papers that Lima-Castillo “clued the government in on a massive fraud conspiracy, with the best of intentions, only to fall victim to the allure of easy money.”

Cuffee, the IRS agent, said that each of those involved in the tax refund fraud business typically plays a specialized role. One person finds and steals the identifying information, another prepares the return and another cashes the refund check — each taking a cut of the proceeds, Cuffee said. And they do it before the identity theft victim files a tax return and realizes something is amiss, she said.

“That obviously becomes a hardship on the legitimate person who is rightfully entitled to a refund,” Cuffee said.

The IRS said in a series of fact sheets that stolen identity refund fraud is a “top priority” and that the agency has assigned more than 3,000 employees to work on ID theft cases. The agency said that from 2011 through November 2013, it stopped 14.6 million suspicious returns involving more than $50 billion in fraudulent refunds.

The problem, though, persists. In the District, federal investigators found that Kevin Brown, owner of Classic Kutz Hair Salon & Barber Lounge, was at the center of a ring of more than 100 people who worked to steal identities and use them to get tax returns they did not deserve. Prosecutors have charged several in the ring, including a former bank teller accused of helping to cash the checks and a former nursing home security guard accused of stealing the identifying information of current and former residents of the home. Brown, who has pleaded guilty, is awaiting sentencing. He and his attorney declined to comment for this report.

Olivo, 29, a father of three whose family owns a cattle ranch in the Dominican Republic and who came to the United States to work as a barber, told a federal judge that he felt “shame” over what he had done because “I was not taught to behave this way.” His attorney wrote in court papers that Olivo will probably be deported once he is released from prison and plans to reenlist in the Dominican army or join his family’s business.

The court papers make it clear that the work will not be nearly as lucrative as Olivo’s trafficking in fake tax returns.

“After all,” his attorney wrote, “his ‘affluent’ background was not enough to keep him from leaving his homeland to live in a rented room in the Bronx and work as a barber.”

Get updates on your area delivered via e-mail