Have you heard the joke about the Secret Service?

The one in which President Obama told a ballroom full of reporters at last weekend’s White House correspondents’ dinner that he had to leave early to get his security detail “home in time for their new curfew”?

Tim McCarthy heard it. He was vacationing on Marco Island, Fla., when Obama’s joke came on the news, replayed again and again by cable channels.

McCarthy, 62, is the former Secret Service special agent who was struck in the abdomen by a bullet from John Hinckley’s .22-caliber pistol March 30, 1981 — right there outside the very same Washington Hilton where Obama made his dig. Look it up on YouTube: McCarthy’s the guy in the steel-blue suit turning his body into the path of the gunshots as his partners, Special Agents Jerry Parr and Ray Shaddick, pushed Ronald Reagan into the waiting limousine, saving the president’s life.

That video clip used to stand for what the Secret Service was all about — before 12 agents and officers were caught boozing and womanizing in Cartagena, Colombia, last month, turning the august institution into a national laughingstock.

McCarthy said he chuckled at the president’s wisecrack, but he acknowledged that it felt like another punch in the gut.

“It bothered me,” said McCarthy, who has been the police chief in Orland Park, Ill., for 18 years. “I’m disappointed something like this happened and personally embarrassed that the service is the butt of jokes.”

He’s not the only one. Across the country, former Secret Service agents are coping with perhaps the most salacious scandal in the 147-year history of an agency whose motto, “Worthy of trust and confidence,” is held as dear as “Semper fi” is by the U.S. Marines.

Long regarded by friends, co-workers and even strangers in the highest esteem, these men and women — whose careers were distinguished by doing everything they could to stay out of the news — are now on the defensive, tarred by association with the agents-gone-wild night of carousing in Cartagena that purportedly included visits to strip clubs and payments to prostitutes.

Congress is demanding answers. The Department of Homeland Security is investigating. Newspapers are questioning the culture of the service. Cable television is giving airtime to critics of the agency. And one of the alleged prostitutes is reportedly considering posing naked for Playboy — or Hustler.

Even Sarah Palin got into the act, responding to a report that one supervisor implicated in the scandal had posted a picture of her on Facebook and joked that he was checking her out from behind.

“Check this out, bodyguard — you’re fired!” Palin crowed gleefully on Fox News.

“It’s like a feeding frenzy, a pile-on,” sighed Pete Cavicchia, a 30-year veteran of the service who now serves as president of the 3,000-member Association of Former Agents of the U.S. Secret Service.

Cavicchia sent an e-mail this week to his members, pleading with them not to talk to the media, although he agreed to talk to The Washington Post to defend the agency’s reputation.

“People who do not have an opinion that counts for anything now have an opportunity to kick us when we’re down,” he said.

There have been more spectacular and consequential Secret Service operational failures — the fatal shooting of President John F. Kennedy in 1963, the near-killing of presidential candidate George Wallace in 1972 and the wounding of Reagan. Agency officials have said that Obama’s security was never compromised, even though at least 11 agents and officers were recalled and replaced by a new team one day before the president arrived.

But nothing has debased the agency’s image quite so thoroughly as the Cartagena misdeeds, not even the time in 1995 when 10 Secret Service agents were investigated for allegedly participating in the “Good Ole Boys Roundup,” an annual gathering of law enforcement officials in Tennessee, where racist acts were said to have occurred.

The bad publicity has prompted a handful of agents to fight back publicly, taking to the airwaves to defend an agency that has suffered 29 deaths in the line of duty, the names of the dead etched in stone on the National Law Enforcement Memorial.

Dave Wilkinson, who spent 22 years in the service, including eight on the presidential protective detail, estimates he has done more than two dozen interviews over two weeks, appearing four or five times on CNN and speaking with a Moscow radio station and a Mexican newspaper.

“I heard a couple of reporters suggest the culture of the Secret Service was something that led to this, that this type of behavior was condoned,” said Wilkinson, now the chief executive of the Atlanta Police Foundation. “That infuriated me. This was a terrible thing for the Secret Service, and we have to take our lumps. But to suggest the culture allows this is absolutely ridiculous.”

Others have dealt with the fallout privately, commiserating with one another, a customary closing of ranks in an agency where privacy is deeply valued. In recent years, agents have been ostracized for writing tell-alls that expose personal details of those they protect, and agents still get grief when they inadvertently appear in news photographs.

But even for those who have kept a low profile, there have been some awkward moments. One former agent, who now operates a private security firm, was scheduled to appear at a seminar about executive protection when he got an e-mail from an attendee asking if “there will be hookers there.”

“He thought it was funny. I had to take it as a joke. I said, ‘No, there will not be because I’m a former agent, not a current agent,’” said this person, who spoke on the condition of anonymity to explain his personal feelings. “But it was a slap in the face. Last year, when I walked around with my credentials, people respected me before they met me. Now, it’s the opposite.”

What hurts most is the blow to the personal pride former agents have for the institution, said Barbara Riggs, who spent 31 years in the service and in 2004 became the first woman named as deputy director.

“Being a Secret Service agent is a lifestyle, not a 9-to-5 job. We call ourselves a family,” she said. Riggs is distressed by the “representation of our culture as ‘Animal House.’ ”

In some cases, the clarion call of the service has been multi-generational. James Huse Jr., 68, a retired agent living in Northern Virginia, has two sons employed by the agency — one as a supervisor on the presidential protective detail and the other as an attorney for Secret Service Director Mark Sullivan.

Growing up with a father who was often away from home on far-flung assignments, Huse’s sons learned about the toll that being an agent can take on a family. The elder boy used to say he would never be an agent, although he has now served 16 years.

During a recent birthday party for one of Huse’s six grandchildren, Huse and his elder son walked Huse’s dog and talked about the disgrace that has befallen the agency.

Huse said his sons are “hurt,” but he was more philosophical, recalling the “Fleet Week” days of his youth, when Navy sailors would descend on his Boston neighborhood for some old-fashioned, boys-will-be-boys fun.

“It’s characteristic of some young people who are not married,” Huse said. “Then maybe they get around a couple guys who organize something because they’re — what’s the term? They’re players.”

But Cavicchia isn’t cutting the Cartagena cabal any slack. A Vietnam War veteran, Cavicchia recalled returning home after the Tet Offensive only to be greeted by a nation that was largely opposed to the war and indifferent to his service.

“To survive that and be treated with scorn and ridicule was heartbreaking,” Cavicchia said. “I thought I’d never see anything like that again in my life. To have it happen again is crushing.”