This post has been updated.

Rep. Justin Amash (R-Mich.) took a public shot at House Speaker John Boehner (R-Ohio) Tuesday night, in the wake of a shake-up that knocked a small group of conservatives off prominent committees. 

Amash, a libertarian-leaning conservative, was taken off the House Budget Committee. Tea party-backed Rep. Tim Huelskamp (Kan.) was taken off Budget and Agriculture. Reps. Walter Jones (N.C.), a rare moderate, and David Schweikert (Ariz.) lost their spots on the Financial Services Committee. Amash, Huelskamp and Schweikert were all first elected in the 2010 Republican wave.

Republican leaders told National Journal that the Republican Steering Committee made the moves after looking at multiple factors, including voting against leadership or committee chairs too often.

At a Heritage Foundation lunch Tuesday, Huelskamp suggested the purge was punishment for being too conservative: “It’s petty, its vindictive, and if you have any conservative principles you will be punished for articulating those.”

Amash said it was his openness to defense cuts that earned leaders' ire. “I think they’re willing to take really bad deals to avoid any defense cuts,” he said of GOP leadership. 

Jones, the only moderate in the group, said that he was now open to voting for a discharge petition, backed by Minority Leader Nancy Pelosi (D-Calif.), to force a vote on the Senate Democrats' fiscal cliff plan. 

“I, at this point, am not going to sign the discharge petition, but I said ‘at this point,’" he told reporters. "I don’t know what next week will bring." 

Schweikert told Roll Call that if the move was an intimidation attempt, it won't work. “You still just work your heart out. You don’t all the sudden become a squish,” he said. “I’m bemused."

Conservative groups have defended the lawmakers and scolded Boehner.

“This is establishment thinking, circling the wagons around yes-men and punishing anyone that dares to take a stand for good public policy,” FreedomWorks president Matt Kibbe said in a statement. The group urged activists to call Boehner and demand he restore the lawmakers to their committee spots.

The Club for Growth crowed that Schweikert, Huelskamp and Amash were "now free of the last remnants of establishment leverage" and promised to help them in future elections.

Tim Chapman, CEO of the Heritage Foundation's action arm, told Slate the move was "very concerning."

RedState's Erick Erickson wrote Tuesday that "conservatives must seek retribution or become the paper tiger John Boehner, Eric Cantor, and Kevin McCarthy just declared them to be." 

On Wednesday afternoon, Boehner denied that the move was a purge aimed at conservatives, saying ideology was not the motivation. He added that the Steering Committee "hopes" it was a one-time thing. 

"This was not done lightly," he said, according to Roll Call. "This is something the committee took seriously and hopes never to have to do again.” 

According to Huelskamp, Boehner told the gathered members, "'we're watching all your votes.'"