Perhaps the first thing ardent boxing fans thought of when they heard “Smokin’ Joe” Frazier died of liver cancer Monday was his knockout left hook. That’s fitting for the black Everyman slugger who punched his way out of his daddy’s South Carolina sharecropper’s fate, winning a gold medal in the 1964 Tokyo Olympics before becoming the undisputed heavyweight champion of the world in 1970.

For the rest of us, what we most remember about Joe Frazier is Muhammad Ali.

We remember Ali calling Frazier “ugly,” an “Uncle Tom” and, especially, a “gorilla.” And even those of us too young to have stayed up to learn the result of a 15-round prizefight in the 1970s recall how black people laughed and laughed at this.

“I’m cringing in my car right now just thinking about it,” says Janks Morton Jr. He visited Frazier in Philadelphia in the early 1970s with his dad, Janks Morton Sr., who trained Sugar Ray Leonard and was close friends with Frazier’s sparring partner, heavyweight champ-to-be Ken Norton. “I can still see [Ali] sitting next to Howard Cosell punching that [rubber] black gorilla, saying, ‘It’s going to be a thrilla in Manila when I kill that gorilla.’ ”

Morton, 48, a documentary filmmaker from Laurel, says that “what Ali represented, that black-power vein, everybody was rooting for him. But we didn’t stop and pause to understand that was a painful period for Joe.”

Leonard, a boxing champ in five weight divisions from 1979 to 1988, remembers Ali’s taunts. “We laughed. It was funny. It’s amazing how you feel now, in retrospect. You go back 40 years and say, ‘Wow! That was really insensitive.’ ”

Leonard says Ali’s insults weren’t personal; they were meant to rattle Frazier. And they did, for much of the rest of his life. Fighters are supposed to be warriors, Leonard says. “People don’t think we have feelings, but we do.”

Cora Masters Barry, former chairwoman of the D.C. Boxing and Wrestling Commission, recalls Frazier and Ali both being in Fayetteville, N.C., for a prizefight in 1989. “We tried to get them together, and Frazier wouldn’t even be in the same room with him. He just wouldn’t do it.” Ali wanted to be friends, Barry says. “He could have been friends the moment after he said it.” But Ali became one of sport’s archetypal hype men, beloved as the people’s champion and legendary, partially on the backs of men who lacked the verbal skills to spar with him.

He’d done the same rhetorical rope-a-dope to Sonny Liston and George Foreman, but Fra­zier took it harder. Frazier made a disparaging remark after Ali lighted the Olympic torch in Atlanta in 1996 and was vilified.

The narrative was created and driven by Ali, says Jelani Cobb, a professor of Africana studies at Rutgers University. Ali was charismatic and pretty. Frazier was blue-collar and rough around the edges, and, looking back on it, “there was something searing and non-evolved about this tall, light-skinned black man calling another black man a gorilla,” Cobb says. With the rest of America, especially black America, co-signing it.

Kenneth Carroll, 53, a Washington native and director of the DC WritersCorps, is most struck by Frazier’s invisibility. “It was almost a Hollywood-like diminishing of Frazier as a person, the way Hollywood often builds one-dimensional foes so the protagonist can be ever more heroic,” he says. “Everything we knew about Frazier came through Ali’s ridicule.”

Ali symbolized defiance to white control, but many in the black community never pondered the brutal irony of Ali saying that Frazier was “so ugly his face should be donated to the bureau of wildlife.” There were even black nationalists in our community, says Carroll, and “nobody said we can’t laugh at this image because this image has been used against us in the past to ridicule all of us.”

That we never imagined Frazier was so deeply wounded by Ali’s insults is part of a larger failure of imagination about men who look like Joe Frazier. Those of us who knew “Smokin’ Joe” mostly in caricature wish he could have hung on for one more comeback — that of a more evolved nation, and a more evolved black community.

On his Facebook page yesterday, Carroll wrote that Frazier had a hard time in his post-fight life, in part because Ali had diminished him but “never went back to lift him up.”

Ali’s ribbing of Frazier was all in the service of the show, the gate, the purse — it earned both men millions. And it earned Ali the eternal enmity of Smokin’ Joe Frazier.