Warren Buffett, the billionaire investor, thinks that J.P. Morgan chief executive Jamie Dimon would be a bang-up Treasury secretary when Tim Geithner steps down in the near future.

“I think Jamie Dimon actually would be, I think he’d be terrific,” Buffett told Charlie Rose on PBS Monday night.

Jamie Dimon: Very good banker. Not necessarily the best Treasury secretary.

There is a long list of reasons a Dimon appointment almost certainly won't happen, and they go well beyond inevitable jokes about whether he would outsource the Treasury’s cash management function to the London Whale. He may be the most successful banker of his age, but Dimon has advocated a lighter touch in regulation of mega-banks, which does not seem to be the direction President Obama is inclined to go. Dimon relishes being in control and may not take well to having a boss, and he has himself said he isn’t interested in the job.

But if Dimon isn’t quite right for the job, who is? What are the qualities that make for a successful Treasury secretary, the things that Obama should be thinking about as he makes a selection? The answers aren’t as obvious as it might seem.

Treasury Secretary is a strange job. It confers immense stature, yet the actual powers attached to the job are surprisingly limited. He (to date, they have all been men) cannot deploy troops like the Defense secretary, or prosecute cases like the attorney general. In many European countries, the finance minister is in charge of the budget and fiscal policy; not so here, as those tasks are assigned instead to the director of the Office of Management and Budget. Hank Paulson had to turn to the Federal Reserve in order to bail out Bear Stearns and AIG back in 2008, because he had no power to rescue them on his own (that changed after passage of the TARP bailout bill later that fall).

The Treasury secretary has formal authority over things like the government’s issuance of debt, its pursuit of terrorist financing, and U.S. representation on the IMF. The IRS and the national bank regulators at the Office of the Comptroller of the Currency fall under Treasury officially, but are independent agencies and he is not supposed to meddle in their affairs. He doesn’t even get to oversee the Secret Service anymore; it was moved to the Department of Homeland Security in 2003.

Oh, and he gets to sign the money. And that’s more or less it. “The Treasury secretary has much less power than the average man or woman in the street might think,” Paulson wrote in his memoir, “On the Brink.”

So the power of a Treasury secretary rests not in his formal powers, but in his stature and skill at persuasion. Do the president and White House staff trust his judgment and instincts on the big questions, such as what tax and spending policies to pursue? Can he set the tone on financial oversight so that the independent regulators at the Federal Reserve and FDIC follow his lead? Is he a sufficiently skilled diplomat as to steer China toward letting its currency appreciate or to persuade European leaders to act more boldly to avert a crisis?

Those are the jobs of a Treasury secretary, and unlike a corporate executive, he must accomplish those tasks without any raw power to deploy. He cannot simply fire the White House budget director, the Federal Reserve chair, or the French finance minister if he isn’t getting his way.

A Treasury secretary needs stature, of course—and Dimon or any other CEO of a giant bank would have plenty of that. But he also needs to be, for lack of a better word, an operator. Some quite good Treasury secretaries haven’t had particularly deep knowledge of finance or economics, but made up for it with skill navigating the halls of power. Think Lloyd Bentsen under Clinton (a former senator), James Baker under Reagan (previously a lawyer and White House chief of staff), and John Connally under Nixon (a governor of Texas).

There are surely CEOs who fit that bill, but the recent track record isn’t great. Both Paul O’Neill and John Snow, under George W. Bush, (the former chiefs of Alcoa and CSX, respectively) are generally viewed as not having been terribly effective. It may be too early to assess the record of Henry Paulson, who left the CEO suite at Goldman Sachs for the job in 2006. In his own memoir, Paulson described some of the challenges of adjusting to the world of Washington after a long career on Wall Street.

For example, when he wanted to negotiate with Democrats over a bill to strengthen oversight of Fannie Mae and Freddie Mac in 2006, he was overruled in a White House meeting. “I was used to dissent and debate,” Paulson wrote. “But I couldn’t remember the last time everyone in the room had opposed me on an issue. I found this frustrating in the extreme.” One imagines long-serving CEOs like Paulson or Dimon don’t encounter rooms like that very often.