Bill Cosby is a beloved icon. So it gave me no pleasure to follow him to the stage at Constitution Hall on May 17, the 50th anniversary of Brown v. Board of Education, after listening to his remarks.

For his philanthropy toward institutions that have worked on behalf of African Americans, Cosby was being honored by the three institutions, including the Legal Defense and Educational Fund, that share responsibility for winning the Supreme Court decision that broke the back of American apartheid. In his acceptance remarks, however, Cosby told the well-heeled, black-tie audience that "the lower economic people are not holding up their end in this deal."

Unlike the story of Brown, Cosby suggested, this was not about what white people are doing to us; it was about what black people are failing to do for themselves. His remarks excoriated poor black people for their failure to actively raise their children, to teach "knuckleheads" proper English and for spending hundreds of dollars for sneakers while refusing to spend $200 for the educational package "Hooked on Phonics." Cosby also spoke of "people getting shot in the back of the head [for stealing] a piece of poundcake, and then we run out and we are outraged." And he wondered why more people from these communities were not incarcerated. "God is tired of you," he quipped, "and so am I."

I knew, even before I reached the stage, that Cosby's comments would be hijacked by those who pretend that racism is no longer an issue and who view poor black people with disdain. So, departing from my own prepared remarks, I embraced the notion of personal responsibility, at the same time calling attention to problems faced by African Americans that are not self-inflicted.

One example is the now infamous Tulia, Tex., drug sting. With no drugs, no money and no weapons recovered, 10 percent of the black population of this small town was arrested and convicted on the word of one corrupt undercover police officer. The sentences ranged from 20 to 341 years. Only after the Legal Defense Fund and other lawyers represented these individuals in post-conviction proceedings were they released.

Predictably, conservatives are applauding Bill Cosby for saying that the problems of the black community stem primarily from personal failures and moral shortcomings. But just as we in the progressive African American community cannot countenance the demonization of poor people, we must not cede the issue of personal responsibility to ideological conservatives. Most poor black people struggle admirably to raise their children well. Parents, including single mothers, work for low wages, sometimes in multiple jobs, to support their families. Recently Cosby recognized this in a press statement in which he emphasized that he was not criticizing everyone in the "black lower economic classes" but intended to issue a "call to action" and to foster "a sense of shared responsibility and action."

Unlike much of the world, we ignore human rights protections against discrimination on the basis of economic status. As a nation, we wage war on poor people in this country, not on poverty. In many ways we are a nation struggling to maintain our moral compass. Violence and dysfunction in poor black communities are under an especially glaring spotlight. But many of the problems Cosby addressed are largely a function of concentrated poverty in black communities -- the legacy of centuries of governmental and private neglect and discrimination.

Cosby's observations about the senseless violence perpetrated within black communities are undeniable. I do not know anyone who does not condemn it. But Amadou Diallo, shot to death in a hail of 41 bullets by New York police, did not steal a poundcake. He and countless other innocent black people have been killed while unarmed in communities in which policing is driven almost entirely by a "war on drugs" that makes all residents presumptive targets.

Following a recent conversation, Cosby and I agreed on this much: To the extent that he is frustrated and angry about the failure of people to be responsible parents, and about senseless crime and violence, I stand with him; to the extent that we continue to be challenged by the systemic issues of race and racism that the Legal Defense Fund has confronted since the days of my predecessor, Thurgood Marshall, Bill Cosby stands with me.

There is no either/or for anyone who truly works in the interests of African Americans and our nation.

The writer is director-counsel and president of the NAACP Legal Defense and Educational Fund Inc.