YOU HEAR IT from Clarence Mitchell, until recently the NAACP's chief man in Washington and now chairman of the Leadership Conference on Civil Rights.

"I was at a Japanese-American national convention in Sacramento about two years ago, and the rhetoric there was just about like you would have in a black meeting," he says. "They were talking about the things they didn't have that were attributable to discrimination against Asians. By the same token, I was at this awards dinner the Italian Americans had last year, and if you listened to one of the principal speeches about not having Italian Americans on the court, you would have thought it was an NAACP member talking about blacks."

You hear it from Louis Nunez, staff director of the U.S. Civil Rights Commission: "If you combine all the constituencies of this commission now - racial minorities, women, the aged, the handicapped, religious groups - you're talking about a majority of the nation. You're talking about maybe reducing the oppressors to 20 percent of the population - 20 percent oppressing 80 percent."

You can, of course, see it for yourself on television, on government forms, in your morning paper, in court battles, in school books, in popular books, almost anywhere: We are in the midst of a rivalry for injustice, a competition in which group after group is in effect saying, "The trouble you've seen? Nobody knows the trouble I've seen."

That is not merely a matter to chuckle over. It is one of the profound legacies of the black civil rights movement's victory a quarter of a century ago now in Brown v. Board of Education. That Supreme Court ruling outlawing state-sanctioned school segregation, followed by other momentous victories on voting rights and public accommodations and job bias and other issues, "stimulated a greater awareness of injustice across the board and helped give birth to many movements patterned after it," remarks William L. Taylor, a longtime civil rights strategist who now directs Catholic University's Center for National Policy Review.

"That's all to the good," Taylor believes, because many racial, ethnic, religious and other groups have legitimate grievances. Mitchell also gives his blessings to the phenomenon, contending that helping one victim helps all. He recalls, for example, his battle in the 1930s against restrictive housing covenants. "The targets of the restrictions were not just blacks. The targets were also Jews, Italians, Poles and especially Asians," all of whom benefited when the restrictions fell.

But the rivalry for injustice also has created deep dilemmas. If most, of us are victims, who are the villains? If most of us feel pain, how much can we feel or believe the pain of others? As the Civil Rights Commission's Nunez puts it, "It's like I had only one meal a day, and somebody else says, 'Look at me - I'm starving.' That doesn't help me feel less hungry."

"There is a tendency today for people to conclude immediately that any action taken against them is based on discrimination," says David Tatel, director of the Health, Education and Welfare Department's Office for Civil Rights. In too many case, he says, "it's complete paranoia."

Consider some of the stranger cases his agency has had to spend time on, citizen complaints passed on by OCR's regional offices but, fortunately, killed in Washington.

In one race discrimination case, according to an internal OCR memo, "a Chicano convicted murderer attending the University of Minnesota on a work-release program was denied the opportunity to run for president of the student body. The school said it was because of his criminal record and his temporary assignment to the school, but the regional office found a violation anyway."

A high school sex bias complaint "would have required the school to flash the name of the girl's basketball team on the scoreboard as many times as you flashed the name of the boy's team," the memo states.

In the handicapped area, covering the mentally as well as the physically disabled, Tatel says a regional office "was about to find that an employer had discriminated against a woman because he had refused to make reasonable accommodation to the fact that she was depressed all the time." That case was killed, too.

Or try a couple of sex discrimination complaints that OCR regional officials took care of before they reached Washington. In what the internal memo calls "The Unclaimed Kotex Case," OCR "interceded and apparently resolved a case in which parents claimed that 5th and 6th grade girls were being harassed because teachers found a sanitary napkin on the floor of the girl's rest room."

In "The Alphabet Mislabeling Case," a school district "was found in violation for (and agreed to remedy) a practice of teaching kindergarteners the alphabet by giving gender designations to the letters" - "Mr. A" and "Miss B" and so on. (Tatel adds that "that's how my kindergartener is learning the alphabet right now.")

These are by no means typical of the gripes OCR gets; they are on the edge, outer-margin reflections of what grieves some Americans or, no doubt in some instances, of what they think they can get away with.

But even without such cases, Tatel says, the multiplying claims are causing serious problems. They "are overloading the process." They "limit our capacity to pick issues that are more important." They "increase federal control dramatically over people who receive federal funds."

Those remarks bear a striking resemblance to positions taken by Tatel's Republican predecessors and attacked at the time by Tatel's civil rights allies. Indeed, when former HEW Secretary Caspar Weinberger proposed to pick priorities rather than investigate all bias complaints pouring into HEW, it was seen as "an attempt to drive a wedge" between competing groups, Taylor notes. "The covert message to blacks and Hispanics was, 'You know women are filing all these complaints, and if we have to investigate all of them we'll be giving them much more attention than we'll be giving you.'" The proposal was killed.

Tatel says he doesn't know whether Weinberger's proposal was the best way to deal with the flood of complaints, "but this agency is being torn left and right by it."

You must make distinctions amid the chorus of claims, many civil rights strategists will tell you. All right, but which distinctions?

Catholic University's Taylor, for example, draws one line: "In some senses, the discrimination against blacks and against Indians was more pervasive and pernicious in that it had the specific sanction of law, that it became more ingrained, and that it may require stronger affirmative acts to remedy."

OCR's Tatel draws another: "People who are poor and minorities, I think, have a much greater claim on government protection than middle-class people who have access to attorneys or other options in life." He is troubled, for example, by rival demands by poor and comfortable women.

"A lot of the Title IX [sex discrimination] complaints we have are from poor women and involve really serious discrimination in vocational education and other areas. But we have many more complaints by middle-cass and upper-class women who are not challenging the ability to get a job from employed women claiming [faculty] tenure. We also get these athletic complaints, a lot of middle-class women wanting opportunities for intercollegiate athletics."

He throws in the necessary qualifier: "Both of those are serious forms of discrimination." But then, with something approaching exasperation: "When you've got a federal agency with a finite amount of time and resources, where does it put them? You can't do them all."

You hear similar frustrations from the Civil Rights Commission's Nunez, who says that "it's very difficult for us to make any judgment about the severity of deprivation." Japanese Americans, for example, have the highest per-capita income of any group in the nation. Indeed, as a group, Asian Americans have done exceedingly well in education and earnings. But, Nunez notes, "They contend that for their education, their level of experience, they're not doing well."

The Civil Rights Commission, in fact, was scheduled to meet with Asian Americans last week, just as it met last month with Seventh Day Adventists, Jews, Scientologists, Black Muslims and other religious groups on the questions of religious observance in prisons and working on Saturday sabbaths.

Nunez is also being pressed by the American Jewish Committee, made up mainly of prosperous Jews, to scrutinize the oil, banking and insurance industries for anti-Semitism. He tells of attending a 1978 annual meeting of that committee, of "talking to this corporate executive and this attorney - and there was this enormous feeling that this community was embattled . . . Someone can, of course, make the case that, "Well, we were in the same situation in Germany in the Thirties.'"

Nunez also met recently with Cuban Americans in Miami, who, he says, "are beginning to feel the bubble has burst for them," though Cubans as a group, again, have done far better economically than Mexican Americans or Puerto Ricans or others of Hispanic origin. And a 1978 issue of the commission's magazine, Civil Rights Digest, was devoted to white ethnics generally, including one article on "The Least Known Minority." That turned out to be the "Rom [Gypsies] in the U.S. and its Schools."

"It's amazing, the sense of injury," Nunez says. Once "it was simple - we knew we had an enemy, all those racists, the governor standing in the schoolhouse door." But "who is right and who is wrong is not that readily apparent any longer." He can't let himself look at it that way, though. "I have to simplify the problems. I can think about these things intellectually, I can debate them, but that could paralyze you."

He is right, of course. You can't just throw up your hands. But anyone who does try to make distinctions, to grasp the seemingly endless issues and demands and histories of each group, to find the villains, will likely feel overwhelmed. You start with European whites who killed Native Americans and go to different Native American nations who killed each other and end up in masses of complexity and ambiguity. After a while you may start to suspect that the bad guy was just the last guy who won.

"The tragedy is that the various groups of casualties in this society are left to fight among themselves," says Cornell University's Ray C. Rist. "The victims are clawing over a dwindling set of resources."

He, of course, is defining victims strictly by an economic yardstick, and he is certainly correct about the tragedy of poor people competing for the same limited government kitty or foundation funds or jobs. There are strains between some blacks and Hispanics at present, for just one example, over U.S. immigration policy. Vernon Briggs, a labor economist long at the University of Texas and now at Cornell, tells of a Houston conference he attended last year in which those strains surfaces:

"The gist of the conversation there was that blacks in Houston, especially black youths and unskilled workers, were bearing the brunt of the competition for jobs with illegal aliens. One of the blacks said, "There's going to be blood in the streets if the Chicano community does not get its act together on this illegal alien question.'"

William E. Pollard, director of the AFL-CIO's civil rights department, remarks: "What you really run into is not just friction between blacks and Hispanics, but misunderstanding and friction between Hispanics and Hispanics. There are tremendous numbers of Hispanics opposed to the flow of illegal aliens, because they ae competitive with them as well as with blacks and whites." Witness the fight of farm labor leader Cesar Chavez against illegal immigration.

In fact, it is not strictly a "group" question, as Clarence Mitchell emphasizes. "I'm on the side of those who feel that we're being unfair in the way we are trying to keep Hispanics out of this country or trying to put some out who are here. Unfortunately, a lot of my colleagues don't agree with me." If other blacks worry about losing jobs to new arrivals, "It's back to what I've said again and again," Mitchell remarks. "When turf is threatened, those who are on the turf defend it . . . I haven't found that blacks act different from anybody else."

In Virginia, for example, the Fairfax Countywide Black Citizens Association led a fight in February against bringing subsidized housing to a subdivision. That prompted an outcry by Richard N. Hamilton, executive director of the National Black Veterans Association, who called it "reprehensible, inconceivable and almost unbelievable" that black homeowners should oppose public housing in their community.

But why does it surprise so many people when members of the same group fight with each other? Why are we so astonished when blacks divide on whether they would like their children to travel long distances for school desegregation? As anthrolopogists will tell you, there have always been as many differences with groups - ideological, social, economic, cultural, religious - as there are between groups.

Fights within groups show up, for example, at the Equal Employment Opportunity Commission: an official tells of a case in which "a black woman here in our Washington local office charged her district director, who was also a black woman, with discrimination based on race and sex." At the Civil Rights Commission, staff director Nunez remarks, "We have blacks in this commission who have filed discrimination charges against other blacks. You can be black and discriminate against another black."

But if fighting within groups should not surprise us, it does add yet more complexity to the immense rivalry between groups. Where is the villain?

Mitchell sees the villain as "a system" that "operates on a doctrine of scarcity. For example, people who are bricklayers . . . don't want too many people in as bricklayers. It's blacks as well. It's a selfish system based on scarcity."

Mitchell doesn't believe we are unable to create enough opportunities for everyone if we wanted to. So despite concern by many blacks that everyone else is shoving to get in the door that they knocked down a generation ago, Mitchell insists each victim must fight for the other.

"Except that we don't all fight together," says Fred Dorsey, acting general counsel of the Civil Rights Commission. "We're all cutting up a diminishing pie. The theory's fine, but the reality doesn't match."