What started as a lighthearted interview between Stephen Colbert and James Franco turned serious on CBS’s “The Late Show” on Tuesday night, as the actor denied accusations of misconduct that surfaced this week on social media.

After Franco appeared Sunday night at the Golden Globe Awards, where he won best actor in a comedy for his role in “The Disaster Artist,” actress Ally Sheedy posted a series of tweets.

“James Franco just won. Please never ever ask me why I left the film/tv business,” she wrote. (In 2014, Franco directed Sheedy in the off-Broadway play “The Long Shrift.”) Sheedy also tweeted, “Why is a man hosting? Why is James Franco allowed in? Said too much. Nite love ya #goldenglobes.”

Sheedy deleted the tweets, but the screenshots made the rounds online. Later that night, another actress sent a tweet that accused Franco of sexual misconduct. She added “Cute #TIMESUP pin James Franco,” in reference to the pins worn by many actors that evening to support the new “Time’s Up” initiative, a legal fund to fight sexual harassment and workplace inequality. Another actress’s tweet directed at Franco said, “Remember a few weeks ago when you told me the full nudity you had me do in two of your movies for $100/day wasn’t exploitative because I signed a contract to do it?”

Fast forward to Franco’s appearance on Colbert’s show, where the two chatted about his portrayal of filmmaker Tommy Wiseau in “The Disaster Artist” and welcomed Franco’s younger brother, actor Dave Franco, on stage for a cameo. Then, as is now becoming a semiregular practice as harassment allegations pour out of Hollywood, the tone of the comedy show changed.

“I mentioned backstage I wanted to talk to you about this, and if you’re okay talking about it, I wanted to ask you about some criticism you got on Golden Globes night,” Colbert started. “Because you were wearing a Time’s Up pin in support of the Time’s Up movement, which has been created by many powerful women in Hollywood to say that the time is up for the abuse, misuse of women both sexually and otherwise, not only in Hollywood but around the country. … You got criticized for wearing that. Do you know why? And do you have a response, do you have anything you want to say about that criticism?”

“First, I want to say I wore it because I do support it. I was so excited to win, but being in that room that night was incredible. I mean, it was powerful. There were incredible voices, and I support it. I support change,” Franco said. He noted the fund’s “50-50 by 2020” goal, which means equality for “people that are underrepresented, women, and people of color, people in the LGBT community” by the year 2020.

After some applause from the audience, Franco segued.

“There were some things on Twitter … I haven’t read them. I’ve heard about them,” he said. “Okay, first of all, I have no idea what I did to Ally Sheedy. I directed her in a play off-Broadway. I had nothing but a great time with her, total respect for her. I have no idea why she was upset. She took the tweet down. I don’t know. I can’t speak for her. I don’t know.”

“The others, look,” Franco continued. “In my life I pride myself on taking responsibility for things that I’ve done. I have to do that to maintain my well-being. I do it whenever I know that there is something wrong or needs to be changed, I make it a point to do it. The things that I heard that were on Twitter are not accurate. But I completely support people coming out and being able to have a voice because they didn’t have a voice for so long. So I don’t want to — I don’t want to, you know, shut them down in any way.”

Colbert asked Franco if there was a way to have this discussion in a format that isn’t social media. “Do you have any idea of what the answer might be to come to some sense of what the truth is so there can be some sort of reconciliation between people who clearly have different views of things?” he asked. “I mean, it’s a big question, but I don’t know how to leave, or to further this discussion.”

Franco paused. “I mean, like I said — the way I live my life, I can’t live if there’s restitution to be made. I will make it. So if I’ve done something wrong, I will fix it. I have to. I mean, I think that’s how that works. I don’t know what else to do,” he said. “I mean, as far as the bigger issues, you know, how we do it. Look, I really don’t have the answers. And I think the point of this whole thing is that we listen. You know, there were incredible people talking that night. They had a lot to say. And I’m here to listen and learn and change my perspective where it’s off, and I’m completely willing and want to.”

On Wednesday night, Franco is scheduled to be a guest on NBC’s “Late Night With Seth Meyers” as he makes the New York press rounds. He was supposed to appear at a New York Times “TimesTalk” event with his brother on Wednesday afternoon, but late Tuesday the event was canceled.

The Times released a statement: “The event was intended to be a discussion of the making of the film, ‘The Disaster Artist.’ Given the controversy surrounding recent allegations, we’re no longer comfortable proceeding in that vein.”

Read more: 

John Oliver and Stephen Colbert are asking the awkward questions about sexual misconduct

They came to disrupt the red carpet ritual. But did celebrities at the Golden Globes actually make a difference?

Golden Globes: Men had little to say about harassment, and it hasn’t gone unnoticed