2013-03-16T161800Z_01_JME107_RTRIDSP_3_USA-REPUBLICANS-CPAC-8500 Ben Carson. (Reuters)

Facing a backlash over his comments on gay marriage, surgeon Ben Carson accused white liberal critics of racism.

"They're the most racist people there are," Carson told radio host Mark Levin on Monday. "Because they put you in a little category, a box: 'You have to think this way, how could you dare come off the plantation?'"

Carson, a renowned neurosurgeon at John Hopkins Hospital, has risen to stardom in the conservative movement after an attention-grabbing speech at February's National Prayer Breakfast. He's considered a possible candidate for president in 2016. But with his increased profile has come increased scrutiny, and Carson been criticized for a recent Fox News interview in which he argued against same-sex unions thusly:

"Marriage is between a man and a woman. No group, be they gays, be they NAMBLA [the North American Man/Boy Love Association], be they people who believe in bestiality, it doesn’t matter what they are. They don’t get to change the definition."

Medical students at John Hopkins University, where Carson is scheduled to give this year's commencement speech, started circulating a letter asking for the invitation to be withdrawn.

The doctor apologized on CNN days later, saying his comments were "somewhat insensitive" but also misconstrued. "I certainly apologize if I offended anyone, because I was not in any way comparing gays with people who engage in bestiality or sexual child abuse," he said. His point, he said, was just that "there is no group that really gets to have a special dispensation" to change the definition of marriage.

Carson has said the likelihood of a presidential bid is "incredibly small." His real interest when he retires from Johns Hopkins later this year, he says, is in a television career.

The surgeon also told Levin he's gotten an overwhelming show of support from "people who believe similarly but are afraid to speak out, because they fear there may be retribution."