President Trump’s nomination of former presidential candidate Herman Cain to the Federal Reserve Board has been controversial even among Republican lawmakers. (Molly Riley/AP/2014 photo)

A swift defection of at least four Senate Republicans has all but doomed Herman Cain’s chances of winning a seat on the Federal Reserve’s board of governors, a striking rebuke to President Trump in his drive to remake the powerful U.S. central bank.

A strong ally of the president, Sen. Kevin Cramer (N.D.), on Thursday joined three other Republicans — Sens. Mitt Romney (Utah), Lisa Murkowski (Alaska) and Cory Gardner (Colo.) — in announcing opposition to Cain’s appointment to the Fed. Republicans control 53 votes in the 100-seat Senate, and losing the support of four members means Cain would need help from Democrats, which appears unlikely.

“If I had to vote today, I would vote no,” said Cramer, a member of the Senate Banking Committee that would consider any nomination.

As word of the Senate backlash moved through the White House late Thursday, the long odds of confirmation became clear, according to two people briefed on the talks. It also became increasingly likely that Cain would not be formally nominated by Trump after all because the Senate support never materialized.

The rapid rejection of Cain — a 2012 GOP presidential candidate — pauses Trump’s effort to remold the central bank into a more political body with outspoken individuals who share his views. It also reflects a growing unease among Senate Republicans with the way Trump has tried to bend the institution to his will in the past six months.

Trump has jawboned Fed officials and pushed them to slash interest rates and flood the economy with cheap money, even though during his presidential campaign he accused the central bank of creating a “big, fat, ugly bubble.”

And furious with some of the Fed officials his advisers pushed on him last year, Trump has completely taken over the process, seeking to elevate two controversial cable news pundits who have loyally supported much of his agenda. In addition to Cain, Trump has said he wanted to nominate conservative economist Stephen Moore to the Fed.

The Fed is the government agency that sets interest rates and regulates financial companies. It plays a major role in setting the direction of the economy, and Trump has blamed Fed Chair Jerome H. Powell for raising interest rates last year in a way that the White House alleges damaged the economy.

The economy grew 2.9 percent last year, but Trump has said it would have grown 4 percent if the Fed hadn’t acted.

Current and former Fed officials have watched the criticism closely, aware that Trump is apt to attack someone he doesn’t like and then move on to something else. But the assault against the Fed persisted, and the efforts to install Cain and Moore sparked even more concern.

“I worry that a perception that somehow politics is gaining a more important role in these decisions will undermine confidence in this nonpartisan institution,” said Donald Kohn, a former Fed vice chair who spent more than 30 years at the central bank. “I think that’s a risk.”

Tracking how many key positions Trump has filled so far

Trump has not formally nominated Cain or Moore. The White House had no comment Thursday about the growing opposition to Cain among Senate Republicans, or whether the administration plans to formally submit his nomination to the Senate.

Trump’s efforts to pressure the Fed come at a time when the central bank is trying to gauge how best to handle a complicated economy. Unemployment is low and economic growth has picked up in the past year, but there are signs that the global economy is slowing down in a way that could affect U.S. businesses and consumers.

White House officials in the past 25 years have largely avoided direct attacks against the Fed, worried about eroding its independence. But Trump tossed that precedent aside last year when he launched repeated blasts against the central bank, with Powell in the crosshairs. He even asked advisers whether he had the power to remove Powell. When he realized he didn’t, he decided to try to stack the Fed’s other board seats with outspoken political figures who would challenge the central bank’s leadership.

Republicans were cautiously supportive of Moore at first, but Cain’s ascent drew alarm. Cain was well known during his presidential campaign for a bumper-sticker tax plan that he called “9-9-9,” but he has been forced to deny multiple accusations of sexual misconduct and harassment.

“I’ve recommended Herman Cain. He’s a very terrific man, a terrific person. He’s a friend of mine,” Trump said last week. “I have recommended him highly for the Fed. I’ve told my folks that’s the man.”

National Economic Council Director Larry Kudlow said Thursday morning that the FBI was conducting a background check on Cain and that he was still the White House’s pick “at the moment.”

Moore, in an interview, predicted Cain would have eventually been confirmed by the Senate once lawmakers spent more time getting to know him.

“I think having someone with business experience on the Fed would be a real breath of fresh air,” Moore said.

Cain once ran Godfather’s Pizza and served as chairman of the Federal Reserve Bank of Kansas City in the 1990s.

Moore, asked about the criticisms leveled against Cain and himself for being political, said “I’m a policy guy and he’s a businessman, and I think most Americans like the idea of putting non-PhD economists, non-academics into the Fed. I think that’s an asset. I think that’s going to be part of the [2020] campaign, that Trump thinks outside the box.”

Still, the swift backlash to Trump’s Fed picks has worried some White House officials, concerned that Trump was picking unnecessary fights.

Trump was undeterred, though, and has been enamored with Cain for months.

Trump recently gathered with generals and other military leaders for a meeting about the Mexican border, according to two people familiar with the chain of events who spoke on the condition of anonymity to freely describe the discussions. At the meeting, which was held in the White House Situation Room, an aide passed Trump a note informing him that Cain was in the building.

Trump summoned Cain to the meeting, and then told the military brass that they needed to come up with a “9-9-9” plan for the border. The joke fell flat.

Senate Republicans have largely been deferential to most of Trump’s nominations throughout the government, and on Thursday they even won some support from Democrats when they confirmed David Bernhardt to be interior secretary.

But their slim control of the chamber makes it nearly impossible to confirm someone who Republicans can’t rally around.

“I don’t think Herman Cain would be confirmed by the Senate, and I think the president would be wise to find someone who is less partisan and more experienced in the world of economics,” Romney, whom Cain ran against for the 2012 nomination, said this week. “I would anticipate voting no.”

Murkowski said she had earlier expressed concerns about Cain that had not diminished. Asked to elaborate, Murkowski responded: “I’m just not a fan.”

Cain did not return a request for comment Thursday. In remarks Wednesday night in Kansas, Cain called his Senate critics “a bunch of yahoos” and said he would not be deterred by criticism, according to the Kansas City Star.

Senate Majority Leader Mitch McConnell (R-Ky.), who has privately encouraged his members to take their concerns about problematic nominees directly to the White House, declined on Thursday to say whether Trump should refrain from nominating Cain altogether.

But he stressed that the White House needs to consider two factors in its nominations: the individual’s background check, as well as his or her prospects of getting confirmed in the Senate.

“Several of our members have had strong feelings about people who’ve been mentioned in newspaper articles,” McConnell said in a pen-and-pad briefing with reporters. “I think that should be helpful to the administration, actually.”

When asked whether he was surprised by Trump’s consideration of Cain for the Fed, McConnell responded: “Surprised? On a daily basis.”

A number of presidents have appointed campaign advisers or administration officials to the Fed’s board of governors, but there isn’t recent precedent for someone as politically adversarial as Cain. Cain started the political action committee, America Fighting Back, last year to help Trump in the midterm elections, has called for reverting to the gold standard, and wants the central bank to slash interest rates.

Recently, America Fighting Back criticized the 12 GOP senators who voted to reject the president’s national emergency declaration to begin border wall construction. The organization recently sent out a fundraising email dubbing those dozen Republicans as “traitors,” according to the liberal media watchdog group Media Matters.

One GOP senator, speaking anonymously to describe Senate Republicans’ sentiment toward Cain, surmised that those dozen senators would not be eager to vote for Cain after those attacks.