The House voted Wednesday to hold former Internal Revenue Service official Lois G. Lerner in contempt of Congress and request a special prosecutor to investigate the agency’s targeting of advocacy groups during the past two election cycles.

Lerner, who headed an IRS division that processed applications for tax-exempt status, invoked her Fifth Amendment right during two hearings, frustrating Republicans and Democrats who want answers.

The 231-187 contempt vote came three days shy of the date when Lerner apologized at a legal conference last year for actions the IRS took against organizations with “tea party” and “patriot” in their names. Her comments marked the first time the agency officially acknowledged using inappropriate screening techniques toward conservative groups.

Days after the event, an inspector general released a report saying the IRS inappropriately targeted tax-exemption applicants for extra scrutiny based on their names and policy positions.

The House voted 250-168 in favor of the measure calling for a special prosecutor to investigate the matter. House Majority Leader Eric Cantor (R-Va.), who pushed for the move last week, has said the IRS’s actions are “too serious a matter to leave to the discretion of partisan political appointees.”

The contempt resolution asks the Justice Department to seek criminal prosecution against Lerner.

Now the U.S. Attorney for the District of Columbia can consider referring the matter to a grand jury for further review. It is unclear how the Justice Department will proceed.

Lerner’s attorney, William Taylor, has repeatedly denied that his client did anything wrong. “Today’s vote has nothing to do with the facts or the law,” he said in a statement. “Its only purpose is to keep the baseless IRS ‘conspiracy’ alive through the midterm elections.”

Democrats on the House Oversight and Government Reform Committee, which is investigating the matter, have argued that Congress cannot legally hold Lerner, who retired from the IRS in September, in contempt because the panel never explicitly overruled her Fifth Amendment assertion or clearly directed her to testify with the threat of contempt.

Republicans counter that the committee effectively overruled Lerner’s refusal to testify when it voted in favor of a resolution saying she waived her Fifth Amendment right by declaring innocence during the first hearing. They say the committee also warned her that she could face contempt charges for refusing to answer questions at a follow-up hearing in March.

Before Wednesday’s vote, the House oversight committee’s top Democrat, Rep. Elijah Cummings (Md.), urged GOP lawmakers to allow a hearing to discuss the contempt matter with independent legal experts. He said in a statement Wednesday that Republicans “took a step backwards in their duty to uphold the U.S. Constitution by voting to strip an American citizen of her Fifth Amendment rights.”

The panel’s chairman, Rep. Darrel Issa (R-Calif.), described Wednesday’s contempt vote as “a step toward a level of accountability that the Obama administration has been unwilling to take.”

Approval of the resolution places­ Lerner among only a handful of federal officials to be found in contempt of Congress in recent years.

Most recently, the Republican-controlled House voted in 2012 to hold Attorney General Eric H. Holder Jr. in contempt for withholding documents related to a failed gunrunning operation known as “Fast and Furious.”During the George W. Bush administration, the Democrat-led House voted to hold White House Chief of Staff Josh Bolton and White House counsel Harriet Miers in contempt after mass firings of U.S. attorneys and allegations that administration officials had sought to politicize the Justice Department.