Attorney General William French Smith yesterday announced a major departure from two decades of civil rights policy, declaring an end to the Justice Department's vigorous pursuit of mandatory busing and strong disapproval of the use of racial quotas in employment discrimination cases.

Both have been "ineffective" and at times unfair as remedies to discrimination, Smith said in a speech in Philadelphia. And both must be replaced by a "more practical and effective approach to the probelm of equal educational and occupational opportunity."

In practical terms, the change in policy will probably mean that the department will no longer intervene in school desegregation cases in favor of mandatory busing as it has done, often decisively, many times in previous administrations.

Justice Department sources said it probably also means that there will be no more advocacy, in the Supreme Court or anywhere else, of affirmative action plans which include quotas or remedies that amount to quotas. In addition, officials are discussing possible amendments to employment discrimination law to make "reverse discrimination" illegal under the Civil Rights Act of 1964.

"Just as we have compromised the principle of color-blindness through over-reliance on mandatory busing to desegregate our schools," Smith said, "we have come perilously close in recent years to fostering discrimination by establishing racial quotas in other areas . . . .

"A bedrock principle" of the Constitution, he said, "is that the government should treat all citizens fairly and equitably."

He stated that the change in policy does not mean the government will not enforce the law, however. "It would be a serious mistake to interpret this change of focus at the remedial level as a signal that the Justice Department will not vigorously prosecute any governmental attempts to foster segregation. We will not countenance any retrenchment here," he said. "We will not permit any of our citizens to be stigmatized by government as the result of their race."

The new approach, Smith said, would mean that the remedies will be designed to fit the "resulting harms actually being suffered today." In school cases, this means "remedies that actually improve the quality of public education."

In employment discrimination, the solutions should not be directed at the "previously advantaged" group, he said, in an obvious reference to whites disadvantaged by affirmative-action quotas.

"If the government had violated the free speech rights of a specific, identifiable group by favoring another group, the effective remedy would be to secure those rights for both groups," he said.

"But it would not be consistent with our traditions or our law to impose free-speech restraints on the previously advantaged group."

The goal "must always be genuinely color-blind state action," Smith said in the speech to the American Law Institute.

"The time has come in America when more can be accomplished by emphasizing the aspirations most Americans have in common irrespective of race; a high quality of education for their children and the opportunity to make the most of their individual abilities."

Officials in past administrations have made similar comments, but the Justice Department has pursued an active civil rights policy nevertheless. The Carter administration filed a friend of the court brief supporting race as a factor in medical school admission in the Bakke case.

It also supported in the courts minority set-asides for contractors in last year's Fullilove case. And the Justice Department for years has been a driving force behind many mandatory busing cases in the lower federal courts.

A Reagan administration shift from those approaches would represent a dramatic turnaround in the federal government's approach to civil rights law.