Larry Summers, the brilliant-but-polarizing former treasury secretary who had been the front-runner to be the next chairman of the Federal Reserve, has pulled out. Here is Summers's letter to the president.

His decision comes after several days in which the political winds were shifting firmly against Summers. Four Democrats of the 12 on the Senate Banking Committee had signaled opposition to confirming Summers, making his eventual confirmation for the job questionable. (Read more here on the mounting opposition to a Summers nomination last week).

"I have reluctantly concluded that any possible confirmation process for me would be acrimonious and would not serve the interest of the Federal Reserve, the Administration or, ultimately, the interests of the nation's ongoing economic recovery," Summers said in his letter to the president, first reported by the Wall Street Journal.

The $4 trillion question now (that is roughly the size of the Fed balance sheet the next chairman will inherit) is what the president does now.

He could appoint Janet Yellen, the current vice-chair, who is respected in markets and academia and has a resume that seems tailor-made for the Fed job. She would also be the first female Fed chair in the 100-year history of the institution. Here is more on Yellen, and why the White House has been less than enthusiastic about making her arguably the most powerful economic policymaker on earth. And while appointing her would in many ways be the safest move, it also would amount to a bit of an embarrassment for the White House. It would have a clear appearance of going for their second choice candidate.

Another option would be the third name that President Obama has mentioned for the Fed chairmanship in meetings with Congressional allies: Donald L. Kohn. Kohn was the No. 2 official at the Fed until 2010 and a key lieutenant of current chairman Ben Bernanke during the financial crisis. Here are the eight things you should know about Kohn.

If, with Summers out of the picture, the president decides he doesn't want to appoint either Yellen or Kohn, who does that leave? Tim Geithner has long been a favored White House choice, but has also sent clear-as-day signals that he does not want the job.

That leaves dark horse candidates: Roger Ferguson. Stan Fischer. Jeremy Stein. Other players to be named later.

The path for the next Fed chair seemed clear as recently as a week ago. Obama was inclined to appoint Summers. He would face some opposition in the Senate, but surmountable opposition. In the last seven days, as an appointment started to appear imminent, opposition to him started to galvanize, and Summers has elected to pull himself out of the running rather than put the president in a difficult spot.

Now we get to see what direction the president wants to go now that his favored candidate is out of the picture.