Former Vice President Joe Biden prides himself on his bipartisan relationships. But on Thursday, Biden’s mild praise of one Trump administration official became a political liability.

At a speech in Omaha, Biden called Vice President Pence “a decent guy.” That statement drew quick criticism from the LGBTQ community, an influential liberal voting bloc.

Pence has aggressively opposed gay rights as a lawmaker in Indiana and Washington. In 2015 as Indiana governor, Pence signed a bill to extend legal protections to business owners opposed to participating in same-sex weddings because of their religious beliefs. Critics argued the bill legalized discrimination based on sexual orientation.

“Mike Pence has made a career out of attacking the rights and equal dignity of LGBTQ people, women and other marginalized communities,” said Chad Griffin, president of the Human Rights Campaign, one of the largest gay rights organizations in the country, in a statement from 2018. “Now as vice president, he poses one of the greatest threats to equality in the history of our movement."

After Biden’s comments, LGTBQ activists quickly took Biden to task. “Mike Pence believes in gay conversion therapy and allowed an HIV outbreak to happen in Indiana,” activist and podcast host Adam Best tweeted. “No, Joe Biden, Pence is not a decent guy.”

Cynthia Nixon, an actor and Democrat who ran for New York governor in 2018, married her wife in 2012 and is the mother of a transgender child. She tweeted: “Joe Biden you’ve just called America’s most anti-LGBT elected leader ‘a decent guy.’ Please consider how this falls on the ears of our community.”

In the past, Biden has defended his praise of Republicans, arguing that being able to reach across the aisle makes him a better lawmaker. Not this time.

In a tweet Thursday, Biden walked back his statements about Pence. “You’re right, Cynthia,” he wrote. “I was making a point in a foreign policy context, that under normal circumstances a Vice President wouldn’t be given a silent reaction on the world stage. But there is nothing decent about being anti-LGBTQ rights, and that includes the Vice President.”

Biden is well-respected within the LGBTQ community for his advocacy of gay rights. He made headlines in 2012 for coming out in support of same-sex marriage before President Barack Obama did. And he was recently the featured speaker at the national dinner for the Human Rights Campaign, one of the more most influential gay rights organizations in the country.

Biden supporters argue that the former vice president is one of the few Democrats who appeals to the white working-class and independent voters who backed Trump in 2016 and the more liberal Democratic base. But figuring out how to woo both groups will be a challenge for the former lawmaker who is considering a 2020 presidential campaign. Thursday’s exchange is likely just a taste of what is to come if Biden enters the race for the Oval Office.