Looking to make amends in the new year? You’re in good company with singer-songwriter Ani DiFranco, who issued a statement Thursday apologizing for her planned artists retreat at a former slave plantation.

“it was a great oversight on my part to not request a change of venue immediately from the promoter. you tried to tell me about that oversight and i wasn’t available to you,” she told her critics on Facebook. “know that i am digging deeper.”

DiFranco, a noted feminist who received an honorary degree for advocating for social justice, canceled her “Righteous Retreat” scheduled for spring and issued several apologies. Nottoway Plantation, the slated venue, is billed as “the largest antebellum plantation in the South” and although it acknowledges its role in slavery, the current owners have been criticized for airbrushing its past.

The criticism of DiFranco is completely justified. She ignored how the location’s history hurt and alienated artists of color. But much of the outcry is limited to a narrow focus. It’s necessary to point out a Confederate flag, a Southern plantation, a white pointed hat or a swastika when it fuels hatred and division. But often it’s easier to dwell on these symbols of the past and the people who misuse them than to acknowledge some awkward truths about modern injustice.

The things we value and consume at the cost of others’ freedom are ugly reminders that the vestiges of racism and slavery linger. As an obvious example, consider that America’s most beloved monuments were also built by slaves. The White House and U.S. Capitol building would also have to be boycotted if the logic used in DiFranco’s situation were applied evenly. The American flag also has a complicated history.

“When people say that the Confederate flag represents slavery, I ask them, ‘If you look at the importation of slaves to the United States from Africa, what flag did they fly under?’ It was the United States flag,”  George Mason University’s Walter Williams told CNN in discussing Kanye West’s use of the Confederate flag on tour merchandise.

Racism and slavery are woven into America’s history but both are more complex and more insidious than the symbols we’ve traditionally assigned them. The media are right to point out misuse of these symbols and ignorance on the part of celebrities. But for all the outrage, disproportionately less is said about the day-to-day instances of injustice. While DiFranco’s critics oppose patronizing a former slave plantation, the tags of most mainstream clothing betray an inconvenient reality.

“It’s really impossible to go a day without benefiting from the work of a slave,” said Ryan Day, a representative of Love146, the anti-slavery and anti-child trafficking organization. “The clothes you’re wearing and the food you’re eating has been brought to us on the backs of slaves.”

So if we strongly condemn these institutions of the past, what do we do about the present? After critiquing celebrities’ misgivings, we can look inward.

“Combating institutional racism is hard but it’s our responsibility to be critical about it and actively resist it,” said Salem State University’s Roopika Risam, an assistant professor of English who also studies race. When you see something racist, “don’t dismiss it,” Risam said. And if you hurt someone, don’t “explain it away.”

“Living a ‘slave-free’ life would be an incredibly radical and extreme move for people and nearly impossible to do in the system and culture that’s set in place,” said Day of Love146. “But we ask that people are more thoughtful about food, clothes and goods they’re buying. It’s more obtainable for people to be thoughtful in their decisions rather than be perfect.”

This all brings us back to Ani DiFranco. We might not be the ones renting out former slave plantations but if we place more importance on symbols and celebrity rather than ourselves and the impact we have on our communities, racism and modern-day slavery will endure under our selective gaze.