"I, Stephen Colbert, have never cared for our president," Obama, pretending to be Colbert and sitting at his desk, said. "The guy's so arrogant, I bet he talks about himself in the third person."
Obama said he "felt more powerful behind" Colbert's desk. But, "I love the job," Obama said of his own.
A buoyant Obama also had a confession: Despite being president, he still leaves his socks on the floor. How does that go over at home? "Not well," Obama said. And no one there treats him like president, he said.
"When I go home, Michelle, Malia, Sasha give me a hard time," he said. "There are no trumpets and they tease me mercilessly for my big ears or my stodgy suits."
Colbert had a question for the president: Is he still the leader of the free world after the midterm elections?
"Because the Republicans are quite surprised that you're doing anything at all," Colbert said.
"Look, the election didn't go as I would have liked," Obama said. "A correction there. I had a little thought bubble."
Obama said he's committed to working, with Congress when possible, to help working families and to make college affordable. He sidestepped a question about the Keystone XL pipeline.
Obama said he thinks young people — and being that the show was taped at George Washington University in Washington there were plenty in the audience — didn't vote in the midterm elections because "they felt discouraged about what was happening in Washington."
Obama touted the Affordable Care Act and the increased numbers of people signing up. Now, he said, the only way for it to be repealed is for Congress to pass a bill to do so, which, posing as Colbert, said would require the president's veto.
"And if I know that guy, he's willing to use it," Obama said.
While interviewing the president, Colbert talked about the improving economy and asked Obama: "Why didn't you fix the economy before the election?"
Obama said the economy has been on a "pretty good run," citing nearly five years of private sector job growth.
"You've employed a lot of people," Colbert said. "Mostly as secretary of defense."
"Well, that boosted our numbers a bit," Obama said.
Colbert did ask Obama what the president deemed said was, for the "first time," a "sensible question" from the comedian. While campaigning in 2008, Obama said that too much power rested in the presidency. Now Obama is issuing executive actions. So do presidents take office and think, "I might be the only one I trust with this much power, so I'll hold onto it?"
Obama said presidents have the tendency to want to get things done, especially when government is gridlocked. He said he uses the White House office of legal counsel to independently advise him on what he can and can't do, but he would prefer to work with Congress.
Colbert said he wasn't going to ask Obama for the nuclear launch codes — but requested a hint.
"Can you tell me if there's a 5 in there?" Colbert asked.
"No," Obama deadpanned.