Whether most Americans agree with Miller or the Republicans who grilled him will go a long way toward determining how bad the episode will be for the Obama administration in the long run.
"I can say generally, we provided horrible customer service here. I will admit that. We did horrible customer service," Miller, the acting agency head who is resigning from his post at the demand of President Obama, told lawmakers.
He explained further: "I think that what happened here was that foolish mistakes were made by people trying to be more efficient in their workload selections."
At issue is the IRS's revelation that conservative groups seeking tax-exempt status were singled out for extra scrutiny. An inspector general's report released earlier this week described the use of “inappropriate criteria” to screen political advocacy groups with certain words in their names, like "tea party" or "patriot."
Miller objected Friday to using the term "targeting" to describe the actions of agency personnel who gave the groups more attention. “When you talk about targeting, it’s a pejorative term,” he told Rep. Charles Boustany, a Louisiana Republican who asked why previous IRS commissioner Douglas Shulman had informed him in the past that there was no such "targeting."
Of course, this is, at some level a dispute of semantics. While "targeting" can connote going after someone with intent, it can also be used to simply suggest that certain entities were singled out.
"I'm going to take exception to the concept of targeting, because it's a loaded term," Miller said later in the hearing.
Republicans didn't agree.
"I know you're disagreeing with the word 'targeting,' Mr. Miller," said Rep. Tom Reed (R-N.Y.). "I suggest the American people will make that determination. And I'll give this whole situation a name; it's the IRS Targeting-gate. I'll put it right out there."
"You say it was not targeting, by why was only one side of the political spectrum singled out in this?" Boustany asked.
Miller said it wasn't only one side, that all groups with a political focus were scrutinized more.
"Look, they get 70,000 applications in there for 150 or 200 people to do," Miller said. "They triage those. People look at them and they send them either through the system because they're OK, into a mix of folks so that they can get technically fixed up and some go for substantive questions. Politics is an area where we always ask more questions. It is our obligation under law to do so."
J. Russell George, the Treasury inspector general for tax administration, also testified before the committee Friday. He used the word "targeted," as did the inspector general's report on the incident that was released Tuesday.
"Our report issued earlier this week addresses three allegations: first, that the IRS targeted specific groups applying for tax-exempt status; second, that they delayed the processing of these groups' applications; and third, that the IRS requested unnecessary information from groups it subjected to special scrutiny. All three allegations were substantiated," George said.
If Americans end up agreeing with Miller's take -- i.e. that this was a misguided effort to make things more efficient -- the scandal could fizzle out fairly quickly.
If they decide that it was something more, it's going to be a long next few months for the Obama administration.
The IRS scandal could hardly have come at a worse time for the administration, with mounting questions about last year's deadly attack in Benghazi and a Justice Department decision to obtain journalists' phone records. Taken together, the incidents have raised questions about the administration's conduct -- questions Republicans have eagerly answered by claiming the developments are evidence the Obama administration acts without regard for rules and the law. If the public agrees, Obama's legacy could be permanently tarnished.