The Reagan administration is wiping out decades of gains in racially integrated housing and moving toward abandonment of public housing after it becomes resegregated, black leaders and integration activitists charged this week during a frequently acrimonious debate with William Bradford Reynolds, the Justice Department's civil rights chief.

James Farmer, a civil rights activist since the 1960s and a former assistant secretary of the old Department of Health, Education and Welfare, said he is "troubled" that the Justice Department is "filing suits against the good people, those people who are trying to create . . . a community of good will and a multiracial society . . . ."

Farmer said that although there are about 2 million cases of discrimination in housing per year, according to the Department of Housing and Urban Development, "the Department of Justice has filed suit against 20," choosing cases "involving those people who have admitted blacks and who are maintaining integrated communities rather than those . . . who have not admitted any blacks." Among the 20 government cases is "Starrett City, where you have not the haters, not the people who are trying to pit race against race, but the people who are trying to maintain an integrated community . . . ," Farmer said.

Starrett City is a giant New York City housing project which uses quotas to maintain a racial balance of 63 percent white residents and 37 percent black, Hispanic and other minorities.

The debate, on the CBS television network's Phil Donahue Show, centered on the use of racial quotas to assure integration in subsidized housing and similar plans in private residential communities. Quotas, and the government's opposition to them, have become explosive issues. Advocates of quotas say that without them, many housing projects and communities would become all black or other minorities.

Reynolds said the Reagan administration believes racial quotas are wrong. " . . . It is also terribly important that we not buy into discrimination in order to get rid of discrimination. If you try to remedy that wrong that is out there by favoring people because of their race and disadvantaging other people because of their race, you're committing the same evil, the same sin, we're trying to get rid of . . . ."

He defended the administration, saying its record "is unmatched by any prior administration in filing suits against those" who totally exclude blacks from housing. He added that "we have over 200 investigations in the housing area alone that are over 90 percent targeted in that area" -- housing where no blacks are admitted.

Farmer, who founded the Congress for Racial Equality and is now president of the Fund for an Open Society, said many in the "civil rights community" suggest the Reagan administration hopes to see subsidized housing become all black. Saying blacks are not Reagan administration constituents, Farmer asked Reynolds, "Is there a game plan to turn subsidized housing into all-black housing and then to yank the funds out from under it, and destroy the subsidized housing field?"

Reynolds said there is no such plan, adding, "Our constituency in the civil rights division are blacks, and whites, and Hispanics, and every other American out there who has the right . . . to an equal opportunity."

Although many civil rights leaders argue that quotas frequently are necessary to maintain racial balance in public housing and private communities, the issue has placed some in uncomfortable positions. The National Association for the Advancement of Colored People, though strongly favoring integration, sued Starrett City because the quotas resulted in black families' having to wait longer for apartments than whites. The NAACP is opposing a quota system in Charlottesville public housing for similar reasons. Charlottesville city officials and the Department of Housing and Urban Development are discussing whether the quotas are a civil rights violation.

In Starrett City, a settlement was reached that preserved the quota system but allotted a larger number of apartments to blacks than before. After the settlement, the Justice Department sued Starrett City, contesting the legality of its quotas.

"It is no different to have the manager of Starrett City stand in the doorway and say to a black who comes in, 'no, you cannot have this apartment . . . you're black and we've reached our quota,' " said Reynolds, than to have "George Wallace standing in the school yard and saying to the black, you cannot come in this school because you are black . . . ."

Starrett City fair housing administrator Vernon Douglas disputed the assertion that "we are not allowing blacks in." In the last 10 years at Starrett City, "kids . . . have grown up in a multiracial environment," he said. "Black kids know that Asian children do not just work in laundries and . . . restaurants. They've played together. White kids are finding out on a day-to-day basis that black people are normal and want the same things. Hispanic kids are finding that out. It is a conscious process to create integration."

Without such an effort to maintain integration, "it's obvious what's happened in subsidized housing over a period of time. You wind up getting an all-black community, an all-Hispanic community, an all-white community," Douglas said. "History shows us that minorities suffer. The other thing that suffers is that people do not learn to live together."

Douglas said "no civil rights organization in the country believes" the Reagan administration is "working hard" to achieve integrated housing as Reynolds contends. "Government has always taken a position up until this administration that integration was important . . . . This administration has stymied the Fair Housing Act."

Some suburban communities also are trying to maintain integration with quota-type systems. Oak Park, a Chicago suburb, which was 1 percent black in 1970, was a pioneer. Now about 13 percent of its 55,000 residents are black and another 5 percent are other minorities. Roberta Raymond, director of a citizens group there, said many groups and organizations worked together to achieve these gains, and now "property values are rising."

A fifth participant in the debate was William North, general counsel of the National Association of Realtors, who said real estate brokers are "caught in the middle between two competing philosophies, one which says . . . race is not a factor in the decision to move or where to move, and the other which says we must have managed integration to achieve some sort of equality . . . ."