House Speaker Paul D. Ryan (R-Wis.) speaks with reporters in April. (Aaron P. Bernstein/Reuters)

Moderate Republicans are directly confronting his leadership over immigration. And on Friday, conservative Republicans called his bluff by helping sink a must-pass farm bill.

It's been a pretty bad week for House Speaker Paul D. Ryan (R-Wis.), who is trying — and right now failing — to hold his fractured caucus together for just a few more months.

Ryan is retiring at the end of the year, and whatever tenuous grip he had on unruly House Republicans over the past three years appears to be slipping as his lame-duck status settles in. After he announced his retirement in April, he had to withstand grumbles from his own party that he was staying on until after the November midterm elections.

Shortly after that, moderate House Republicans started a petition, over Ryan's objections, to force the House to vote on a series of immigration bills. GOP lawmakers — 20 of them  — have signed onto the petition even as Ryan and his leadership team warn that the path they're going down could jeopardize the House Republican majority.

House moderates normally are not the problem group for Ryan, points out Molly Reynolds, a congressional expert with the Brookings Institution.

He's been able to count on this group to be the reliable faction, the ones who will take tough votes and won't complain publicly about doing it. But that has changed as a growing number of lawmakers get desperate about their own election chances. They want to be seen taking action to protect “dreamers” before they face voters in districts that Democrats will target this November. At least four signatories are retiring, suggesting that when they have the freedom to speak their mind, they would rather buck Ryan on this.

Then, on Friday, a more familiar standoff vexed Ryan. House conservatives threatened to vote against a must-pass farm bill (which does a lot in addition to providing subsidies to farmers) if Republican leaders didn't bow to their demands on immigration.

Ryan didn't and decided to bring the bill up for a vote anyway. It was a gutsy move on his part. Kinda like lawyers don't ask questions they don't already know the answer to, House leaders rarely bring a bill to the House floor without knowing what will happen, the better to avoid failures (and headlines) like this.

But a bloc of 30 conservatives refused to back down and joined all House Democrats to vote against the bill, killing it.

That means that in one week, Ryan is 0-2 in battles with his caucus.

There's always the possibility he salvages both issues. Ryan already restarted the vote process for the farm bill, hoping last-minute negotiations with conservatives provide a solution in time.

That strategy prevailed last summer as the House narrowly passed a repeal of Obamacare after first voting against it. That vote is now top of the list of things Ryan brags about as his accomplishments as speaker. (The Senate voted down the bill by one vote.)

Ryan and his leadership team have also reopened negotiations with moderate House Republicans to vote on immigration bills, against their better judgment.

Of course, Ryan is leaving in a few months, so his lame-duck struggles won't last forever.

But nothing we've seen this week suggests any of this potential successors, like his preferred choice, House Majority Whip Kevin McCarthy (R-Calif.), can quell the increasingly dramatic uprisings happening in Ryan's final months on the job.