John Kelly, former chief of staff to President Trump, said Saturday he warned the president before he left the White House not to replace him with a “yes man” because it would lead to Trump’s impeachment.

Kelly also said he believed he could have prevented the current impeachment inquiry against Trump if he had stayed in the job. He said the inquiry could have been avoided if the president had surrounded himself with people who could rein in his worst instincts.

His candid remarks, made during an interview at a political conference hosted by the Washington Examiner, suggests he blames acting chief of staff Mick Mulvaney and others in the West Wing for not doing more to stop Trump’s behavior.

“Someone has got to be a guide that tells [the president] that you either have the authority or you don’t, or Mr. President, don’t do it,” Kelly told the Washington Examiner’s Byron York. “Don’t hire someone that will just nod and say, ‘That’s a great idea, Mr. President.’ Because you will be impeached.”

The House Democrats are in the midst of an impeachment inquiry involving Trump’s request to Ukrainian President Volodymyr Zelensky in a phone call to investigate former vice president Joe Biden, a 2020 presidential contender, and his son Hunter Biden.

The White House on Saturday released a statement from Trump denying that Kelly ever delivered that warning, in his first public rift with a former chief of staff.

“John Kelly never said that, he never said anything like that. If he would have said that I would have thrown him out of the office,” Trump said. “He just wants to come back into the action like everybody else does.”

White House Press Secretary Stephanie Grisham added, “I worked with John Kelly, and he was totally unequipped to handle the genius of our great President.”

Kelly, a retired four-star Marine general, has likened his time in the White House to wrestling with a bear every day, according to a person who spoke to him. The person asked not to be identified to speak candidly. Kelly also complained about the president’s daughter Ivanka Trump and her husband Jared Kushner, the person said, as well as what Kelly said was Trump’s obsession with news coverage of himself.

Kelly and Trump were barely on speaking terms when the former chief of staff left — after 18 months of sometimes titanic clashes.

In one of their final conversations, according to a person briefed on the topic, Trump asked Kelly not to write a book — and Kelly agreed, at least until Trump was out of office.

During the interview Saturday, Kelly expressed some regret about leaving.

“That was almost 11 months ago, and I have an awful lot of, to say the least, second thoughts about leaving,” Kelly said. “It pains me to see what’s going on because I believe if I was still there or someone like me was there, he would not be kind of, all over the place.”

Mulvaney has taken the different approach of letting “Trump be Trump.” The acting chief of staff regularly leaves the West Wing to visit his home in South Carolina, or golf, or attend political events, according to current and former aides.

But Kelly had his own controversial episodes in the White House, defending the administration’s family separation policy, making false statements about a U.S. congresswoman and mishandling allegations against Rob Porter, the former staff secretary.

During a second panel discussion, Kelly called Trump’s decision to pull U.S. troops out of Syria a “catastrophically bad idea.”

“It didn’t happen while I was there — and a couple of other people recently left the administration and then he went with his instinct,” Kelly said.