He quit the party in June 1941, after the Nazi invasion of the Soviet Union led the U.S. party to switch gears into building American opposition to fascism, and racial justice issues fell by the wayside. By the end of his life, Rustin was the chairman of Social Democrats USA, the pro-Vietnam War successor party that emerged out of the Socialist Party's collapse in the 1970s, and which was a breeding ground for many neoconservatives (indeed, the word "neoconservative" was first used as a term of abuse for members of the Social Democrats USA). Rustin opposed the war but also opposed withdrawing without a negotiated settlement. As late as 1975, Rustin sent President Gerald Ford a letter urging him to "do whatever is possible to secure the freedom of Vietnamese whose lives are now threatened by the communist military victors."
Despite that link, Rustin was a deeply committed pacifist, owing largely to his Quaker background, and got thrown in jail for conscientiously objecting to service in World War II. He was the one who introduced Gandhi's tactics of nonviolent resistance (which he learned from visiting independence activists in India) to the Rev. Martin Luther King Jr. King's links to Rustin occasionally caused him trouble, not just because of his dalliances with Communism but because Rustin was openly gay, an astonishing fact at the time. Both segregationists and other civil rights leaders would use this against him and King, as Steven Thrasher explains:
In 1960, Rustin and MLK were preparing to lead a boycott of blacks outside the Democratic National Convention. This would have deeply embarrassed the leading elected black politician of the day, Rep. Adam Clayton Powell. Powell threatened to spread a rumor that Rustin was having a sexual relationship with King. King canceled the protest, and Rustin resigned from the Southern Christian Leadership Conference. […] Rustin’s sexual arrest record terrorized him again in 1963, when segregationist Sen. Strom Thurmond read its entire contents into the congressional record, in an attempt to make the march lose its best organizer. It backfired. Civil rights leaders, taking an “enemy of my enemy is my friend” approach, were not supporters of Thurmond and backed Rustin.
Rustin didn't live long enough to see legal recognition for relationships like his and Naegle's, so he creatively opted to adopt Naegle as his son, which allowed Naegle to inherit Rustin's belongings upon his death (which, given that Rustin was 37 years older, was an important consideration). While there was barely a gay rights movement to speak of when Rustin began his activism, toward the end of his life he became more involved, saying on one occasion, "The question of social change should be framed with the most vulnerable group in mind: gay people.
But he always resisted the notion that his activism was driven by personal motives. "My activism did not spring from my being gay, or for that matter, from my being black," he wrote in a letter. "Rather it is rooted, fundamentally, in my Quaker upbringing and the values that were instilled in me by my grandparents who reared me. Those values are based on the concept of a single human family and the belief that all members of that family are equal.
It's important, today, to remember the incredible impact that Dr. King had on Aug. 28, 1963. But it's worth taking a moment to remember the man behind the event, who hasn't been given his due because of decades of prejudice.