As president and during four decades in business, Donald Trump has built his brand by promoting himself as someone who never backed down. When he was hit, he often said, he’d hit back a hundred times harder.

But at pivotal moments throughout his career, when confronted by people wielding equal or greater power, Trump has proved to be someone who does back down.

This week, when House Speaker Nancy Pelosi (D-Calif.) pulled her invitation to President Trump to deliver the State of the Union address in the House chamber this coming Tuesday, the faceoff between congressional leader and president seemed to portend a rift that could extend well beyond the government shutdown.

Trump’s first instinct was to double down on delivering the speech, as he told Pelosi in a letter Wednesday, “on time, on schedule, and very importantly, on location!”

But hours later, the president retreated: Postponing the speech “is her prerogative,” he tweeted. “I will do the Address when the Shutdown is over.”

Trump has always painted himself as an eager combatant against anyone who says no to him. And his ability to hold grudges against those who reject him is legendary.

“There are people — I categorize them as life’s losers — who get their sense of accomplishment and achievement from trying to stop others,” he said in his 1987 bestseller, “Trump: The Art of the Deal.” “As far as I’m concerned, if they had any real ability they wouldn’t be fighting me.”

In 1976, just as Trump was about to land his first development deal, a powerful person told him no in a very big way. Trump had crossed the bridge from his father’s real estate empire in New York’s outer boroughs into Manhattan, where the young developer got a contract to buy the Commodore Hotel building, a sad structure near Grand Central Terminal.

Now he had to pay for it. He approached Richard Ravitch, head of a state agency that had the power to give him a huge tax break. Sorry, Ravitch said: The hotel should be able to succeed on its own, without an assist from the taxpayers. No tax exemption.

Angry, Trump lashed out at Ravitch. “I’m going to have you fired,” Trump said, storming out of the office, the agency head later recalled.

Trump eventually won his tax exemption, thanks to one of his first big PR stunts: He had construction workers cover the hotel’s clean windows with dirty scrap wood to make it look like a dangerous eyesore in desperate need of rehabilitation — by Trump, with the help of public dollars.

Still, Trump never forgot Ravitch’s rejection. Five years later, Ravitch got a call from New York Mayor Ed Koch. “What did you do to Donald Trump?” the mayor asked. “He wants me to fire you.”

At other times, Trump has been much less pugnacious. In 1990, deep in debt and with no clear way to crawl back to solvency, Trump was summoned to a middle-of-the-night showdown with his bankers, accountants and attorneys. He stood to lose much of his casino empire.

At first, Trump was his usual bold and bragging self, insisting that he didn’t need a bailout, that he would rally and win. Then, suddenly, he dropped the insults and boasting. People in the room said he became quiet, respectful, apologetic.

He argued that the banks needed him — his name, his celebrity were what gave his properties value. The bankers agreed to bail him out, but on their terms. They put him on a monthly allowance, put liens on his house and his yacht.

But Trump got what he wanted most: He saved face. His name stayed on the buildings.

He declared victory.

In the confrontation with Pelosi, Trump plays two roles — personal and presidential. On a personal level, his lifelong determination never to be seen as a loser pressed him toward a combative stance, leading him to declare that Pelosi “doesn’t want to hear the truth” and that he would push ahead with the speech.

But as president, Trump was hemmed in by law, tradition and political reality. So when he tweeted late Wednesday that he would yield to Pelosi’s prerogative and postpone the address, the move came as no surprise to longtime Trump watchers and scholars of the presidency.

“He caves when people are tough with him,” said Jeffrey Tulis, a government professor at the University of Texas at Austin who studies presidential rhetoric.

He pointed to the language Trump and Pelosi used in their Oval Office confrontation last month, when Trump called the speaker “Nancy” and needled her over what then seemed like her precarious hold on House leadership.

Pelosi sharply replied, “Mr. President, please don’t characterize the strength that I bring to this meeting as the leader of the House Democrats.”

Tulis said: “She called him ‘Mr. President,’ and she’s remained formal in her letters to him. People talk about how Pelosi is throwing down the gauntlet, but pay attention to how carefully written and institutionally respectful these letters are.”

The effect of the formal language in Pelosi’s letters to Trump about the State of the Union address has been to constrain the president’s behavior, Tulis said: “It has forced him to act more presidential.”

Presidents and speakers of opposing parties can make life miserable for each other. They can block each other’s agendas and generally gum up the works. Or they can reach an arrangement that gets stuff done, as President Ronald Reagan and Speaker Thomas “Tip” O’Neill famously did in the mid-1980s, after they had spent a couple of years going at each other, hard.

Reagan attacked O’Neill’s “sheer demagoguery,” and the Massachusetts Democrat returned fire, saying the president had “no compassion for the poor.” But in 1982, they came together and agreed to raise taxes and overhaul Social Security.

The two went out of their way to portray their partnership as a genuine friendship, but aides to both men said the relationship was more a delicate balancing of political power than a personal embrace.

Similarly, the emerging Trump-Pelosi relationship, now that she and her fellow Democrats have gained control of the House, is more about the battle between the executive and legislative branches of government than it is about two people who have little in common.

“The question of when and where the State of the Union address is given is not hugely important, but the testing of the balance of power between the branches is,” said Joanne Freeman, a historian at Yale who has written a book, “The Field of Blood,” on moments when tempers in Congress have led to violence.

“In this time when so many norms are being tested, we’re seeing two branches asserting power and now Congress claiming, as it should, that it is a coequal branch of government,” she said. “This is really in a very concrete way the will of the president against the will of Congress.”

The struggle between presidents and Congress goes back to the nation’s beginnings. President George Washington knew the Constitution required him to get the advice and consent of the Senate before signing any treaties, but he didn’t know what that directive actually meant he should do.

So when Washington made a deal with Indian tribes, he walked the text over to the Senate and read it aloud to the senators. When one senator asked the president to read the treaty again, and then said the Senate would have to take some time to think about it, “Washington gets upset and storms off and never returns to the Senate to seek advice and consent again,” Freeman said.

More recently, Congress has ceded significant authority to the presidency. Republicans and Democrats have called for Congress to reassert its power. So Pelosi’s decision to draw a line and bar Trump from delivering the State of the Union while the shutdown is still paralyzing the government may win some quiet support from Republicans, Tulis said.

“Many Republicans now are from districts where they will be besieged if they stand up clapping for the president in the middle of a government shutdown,” Tulis said. “They may not say it in public, but many of them are pleased to see Congress reasserting its authority.”

Trump, like those congressional Republicans, is highly unlikely to cede any rhetorical ground to Pelosi. And even if he stands by his position that he will delay the State of the Union until after the shutdown, he will almost surely find another way to score points against the speaker.

After all, public battles have always been the lifeblood of his branding strategy, which is in turn the core of his approach to business. People who have said no to Trump have learned through the years that he never forgets and that he often finds a way to score late, if petty, wins.

By the mid-1980s, Trump and Koch, two of New York’s most notorious mouths, had been sniping at each other for years. The developer called the mayor a “moron” and a “disaster.” Koch squawked back, calling Trump “piggy, piggy, piggy.”

In 1986, Trump saw a way to ingratiate himself with the public and diss the mayor at the same time. From his Trump Tower office, the developer could look down at the city’s long-shuttered ice-skating facility, Wollman Rink.

For six years, the city had tried and failed to reopen the rink, once a municipal gem in Central Park. After $13 million had been wasted on failed repairs, Trump offered to fix and reopen the rink in four months, free of charge — if he was then allowed to manage the facility and name it for himself.

Koch accepted the repair offer but said no to Trump managing the facility or giving it his name.

Undeterred, Trump got the job done way ahead of schedule and under budget — and the city ended up paying the bill. At the grand-reopening news conference, he posted a large sign: “Owner: TRUMP ICE INC.” The city parks commissioner ordered his staff to take down the sign.

Trump portrayed the episode as a single-handed triumph, even though the repair of the rink was well underway before he got involved. Trump’s version became the standard narrative, and his popularity soared.

For years after the rink reopened, Trump pummeled the mayor in one TV interview and newspaper story after another. Koch accused Trump of being a serial exaggerator and called him a “supreme egotistical lightweight.” But mostly the mayor backed away from further rhetorical battle with the developer.

Today, the facility is still technically named Wollman Rink. But the signage and marketing materials tell a different story. “Wollman” appears in light, thin lettering, dwarfed by the big, bold, bright red “TRUMP.”