Gloria Brown grew up in the town of Stone Mountain, Georgia, the birthplace of the modern Ku Klux Klan. Other African American families pulled the shades down or shuttered their windows as men belonging to the hate group, their faces covered by white masks and hoods, marched through town. But Brown’s father refused to hide.

“I was never taught to be afraid of them,” she says.

Brown, who is 78, and other longtime residents of Stone Mountain live with a reminder of that time: a huge Civil War memorial of Confederate heroes that was carved into the side of the mountain for which her town is named.

It features Generals Thomas “Stonewall” Jackson and Robert E. Lee, along with Jefferson Davis, who served as the president of the Confederacy.

Many people want the carvings erased, saying they honor slavery and are offensive to blacks. Brown, whose family history is tied to the memorial, isn’t one of them. She considers the carvings an important reminder of the life she has lived and the racial discrimination that shaped it.

Her mother was a summer maid and cook in the mansion owned by James R. Venable, a white supremacist who led a KKK group for more than two decades. His uncle owned Stone Mountain and tried for years to get the memorial completed.

“Some members of my family worked for the Venable family until I was almost grown,” she says. She remembers getting to look inside the “Big House” when she came to visit her mother or uncle, who was Venable’s handyman there. “It had a winding staircase, and it was very pretty.”

Brown was 5 when her mother died, leaving her father to raise her and her older sister during segregation and the long civil rights movement.

She remembers when the drugstore in town, where her father took her for ice cream, was forced by law to desegregate. To keep black customers from sitting down and enjoying ice cream with white customers, the owner removed the tables.

And in nearby Atlanta, blacks who shopped at Rich’s Department Store had to go to the basement to eat because they weren’t allowed into its Magnolia Room restaurant.

“I was always taught by my daddy, that was just the way of life at that time and it was going to get better,” says Brown, who went on to work more than 40 years for two Atlanta department stores.

She still lives just minutes away from the Stone Mountain monument and only blocks away from a street named after that prominent Klan family — Venable Street.

When Brown was a child, she would climb the mountain, often playing with her friends near the partially completed monument. As a mother, she would take her daughter to see the memorial — which officially opened in 1965 — and explain what it was and why it was controversial.

Like her father, she taught the girl to look ahead.

“I have no ill feelings. It is part of history,” Brown says. “I think about so many black people that worked around that monument. I have no problem with it staying there.”