In a significant expansion of constitutional protections, the Supreme Court yesterday struck down laws denying free public education to the children of illegal aliens, calling the measures an "affront" to the principle of equality.

In a 5-to-4 decision affecting 3 million to 6 million undocumented aliens in the United States, Justice William J. Brennan Jr. called it unfair and unconstitutional to impose "a lifetime of hardship" and "the stigma of illiteracy" on children because of the misconduct of their parents.

It "does not comport with fundamental conceptions of justice," Brennan wrote for the majority.

The decision was attacked by Chief Justice Warren E. Burger in a heated dissent as "another example" of an unwarranted exercise of judicial power, another "trespass on the assigned functions" of the elected branches of government. The ruling has the immediate effect of nullifying tuition requirements imposed in Texas in response to the influx there of thousands of Mexicans. One of the two tuition cases decided yesterday was brought on behalf of children in Tyler, Tex., who were asked to pay $1,000 a year--a prohibitive price--to attend school.

Locally, Prince George's County, Md., also has a tuition requirement, which must now fall. The decision also prevents the future denial of free public schooling in any other state.

Brennan said it was "not merely some governmental benefit" that was being denied in Texas, but education, which, he stressed, "has a fundamental role in maintaining the fabric of our society."

Despite this distinction, lawyers and other observers said the ruling nevertheless raises questions about the continued denial of such "benefits" as welfare, food stamps and medical care to illegal aliens when the country is attempting to cope with a wave of undocumented Mexicans, other Hispanics and Haitians. All the benefit denials are predicated on the idea put forward by Texas and rejected yesterday by Brennan: That people without a right to be in this country have no right to enjoy many of its fruits.

The court said the fact that the losers in Texas were "innocent children" also was a major factor in the outcome of the case.

The ruling was a departure from the court's recent inclination to defer to legislative policy judgments and to deny significant expansions of constitutional protection.

While prior court rulings have provided illegal aliens with constitutional rights of due process, such as hearings before they are expelled from the United States, yesterday was the first time the court had granted them a concrete benefit.

Justices Thurgood Marshall, Harry A. Blackmun, Lewis F. Powell Jr., John Paul Stevens and Brennan made up the majority in the case, Plyler vs. Doe. Justices Byron R. White, William H. Rehnquist and Sandra Day O'Connor joined Burger in dissent.

Powell appeared to make the difference in the voting. He wrote the 1973 decision in San Antonio School District vs. Rodriguez holding that education was not a fundamental right guaranteed by the Constitution. The price of Powell's vote yesterday may have been Brennan's careful statement in the majority opinion that yesterday's ruling does not alter that 1973 decision.

Texas, which borders Mexico, has been flooded with illegal aliens seeking work, so much so that they are now considered a staple of cheap labor in the state's work force. Officials now estimate that 750,000 live in the state, about 5 percent of Texas' population, and that about 20,000 of them are children attending public schools. They have not been paying tuition since 1978 when lower courts struck down the requirement.

Specifically, the courts held unconstitutional a 1975 state law withholding state funds for public education of illegals. The law allowed local communities to exclude the children or charge them tuition.

Brennan rejected Texas' argument that the illegal aliens were somehow not the persons within the "jurisdiction" of the states described in the Constitution as beneficiaries of the 14th Amendment. Everyone within a state is protected, he said, however they got there.

That means they must be treated the same as everyone else, Brennan said, unless there is a "substantial" governmental reason to treat them differently. In Texas, he said, neither the savings to taxpayers, a claimed need to preserve scarce educational resources, the deterrence to illegal border crossing or any other reason put forward was sufficient to justify the denial of a public education.

"Public education is not a 'right' granted to individuals by the Constitution," Brennan said. "But neither is it merely some governmental 'benefit' indistinguishable from other forms of social welfare legislation."

As expected, Hispanic groups praised the decision while supporters of strong immigration controls criticized it as providing a lure to further illegal migration and encouraging new legal efforts to extend welfare and medical benefits. Illegal aliens now are excluded from such benefit programs as food stamps, Medicaid, Medicare, unemployment compensation and welfare.

Because yesterday's ruling concerned state governments and, specifically, a benefit clearly considered special by the court--education--it will not be clear for some time whether it can be extended to these other areas.

Peter Roos, an attorney for the Mexican American Legal Defense and Education Fund, who argued the case, called the decision "a giant step forward in recognizing the positive contributions undocumented aliens make in our society."

Rick Gray, executive assistant to Texas Attorney General Mark White, said the decision shows that "either the federal government should enforce the immigration law or help pick up the expenses states face when it fails to."

The case is expected to have little immediate practical effect on the schooling of alien children because Texas is the only state to have a law barring their free education.

But Irma Herrera, a MALDEF attorney, said the decision was crucial to stopping passage of similar state laws. "Had we lost, many other states would have attempted to pass legislation," she said.

Barnaby Zall, executive director of the lobbying arm of the Federation for American Immigration Reform, said the decision makes enactment of strict new immigration controls by Congress more important than ever. "Providing an education acts as an incentive for illegals to bring their families," he said, "so it's an incentive for extended illegal immigration."

Congress is considering a bill sponsored by Sen. Alan K. Simpson (R-Wyo.) and Rep. Romano L. Mazzoli (D-Ky.) that would give amnesty to millions of illegal aliens but make it illegal for employers to hire undocumented aliens in the future.

Justices Marshall, Blackmun and Powell, who agreed with the Supreme Court's majority ruling, also filed separate concurring statements offering additional views.