The Supreme Court, striking down the Reagan administration's "Baby Doe" rules, said yesterday that the federal government cannot force hospitals to treat severely handicapped infants over the objections of their parents.

The 5-to-3 ruling came in an emotional battle in which the administration allied itself with disability-rights groups on one side as the American Medical Association and other medical groups supported the other side. The administration had championed an active federal role in the matter as part of its social agenda.

In an opinion for four justices, Justice John Paul Stevens said that the federal law banning discrimination against the handicapped does not give the administration the power to intervene directly in decisions about the treatment of such newborns.

Chief Justice Warren E. Burger agreed with Stevens' conclusion but did not join his reasoning.

Stevens' opinion was joined by Justices Thurgood Marshall, Harry A. Blackmun and Lewis F. Powell Jr. Justices Byron R. White, William J. Brennan Jr. and Sandra Day O'Connor dissented. Justice William H. Rehnquist did not participate in the case because his son-in-law's firm works for the American Hospital Association.

The law "does not authorize the [secretary of health and human services] to give unsolicited advice either to parents, to hospitals, or to state officials who are faced with difficult treatment decisions concerning handicapped children," Stevens said. The ruling leaves intact the traditional role of the states in such regulatory matters.

The bitter legal dispute began in 1982 when a Bloomington, Ind., infant known only as "Baby Doe" was born with Down's syndrome and a blocked esophagus and was permitted to die.

The Reagan administration issued rules to force hospitals to treat severely handicapped infants even when parents objected to the treatment. Those federal rules, which Stevens said were based on a "manifestly incorrect perception," were struck down by lower courts.

The administration then adopted similar regulations in 1984 that included a requirement that hospitals provide medical records to federal investigators in cases involving such infants. That second case in New York was called the "Baby Jane Doe" case.

Yesterday's 5-to-3 decision leaves standing a federal appeals court ruling in that case that denied government access to those records and leaves intact a nationwide injunction against any direct federal involvement in treatment decisions.

The administration had presented "no evidence," Stevens said, that would justify federal intervention in an area traditionally controlled by the states.

The federal government has no power to overrule parental decisions, he said, and there was no evidence of a single instance where hospitals denied treatment when parents requested it.

The federal oversight and investigative powers under the rules were impermissible, Stevens said. "The child protective services agencies are not field offices of the HHS bureaucracy," he said, "and they may not be conscripted against their will as the foot soldiers in a federal crusade."

Stevens said "the administrative record demonstrates that [HHS] has asserted the authority to conduct on-site investigations, to inspect hospital records, and to participate in the decisional process in emergency cases in which there was no . . . basis for believing that a violation of [the 1973 law] was about to occur."

"The district court and the court of appeals," he said, "correctly held that these investigative actions were not authorized by the statute and that the regulations which purport to authorize a continuation of them are invalid."

Benjamin W. Heineman Jr., an attorney for the AMA, said the ruling meant the court had concluded that "Big Federal Brother should not be in infant nurseries. The administration has been trying for three years to make this into a federal case and to make a record that there is a problem in care of newborns that justifies a federal role and the court flatly said no."

"I would hope the war is over," he said, and the discussions over treatment issues would now take place in hospital ethics committees, "where they belong."

But organizations promoting the rights of the disabled, citing the lack of a clear majority for Stevens' analysis, vowed they would continue to fight for federal protection in this area through new legislation.

Thomas Marzen, general counsel of the National Legal Center for the Medically Dependent and Disabled, said the court was "trying to hide behind parental skirts" in the ruling, when "everyone knows that physicians, not parents, are really making the decisions."

Disability rights groups, he said, would almost certainly push for legislation that would clearly make such medical treatment decisions subject to federal law.

"We wished the decisions had come out the other way," said Justice Department spokesman Terry Eastland, adding that he saw some indications in Stevens' plurality opinion that some regulations in this area could be upheld. He added, "But I don't know whether HHS or the White House will want to revisit the issue" by issuing new regulations.

Stevens' opinion, though harshly critical of the administration, appeared to focus narrowly on the specific regulations involved in the case. But a dissenting opinion argued that the ruling seemed broader than that.

"I have some doubt," Justice White said, "as to how different the court's holding today is from a holding that [the 1973 act] gives HHS no authority whatsoever over decisions to treat handicapped infants." White said Stevens' opinion, which he called a "misguided effort," "apparently would enjoin all enforcement actions by the [federal government] in situations in which parents have refused to consent to treatment."

Justice O'Connor, also in dissent, said she thought the federal government could regulate some medical treatment decisions but said she did not feel the court should decide now whether the current rules were appropriate.

Only five justices were on the bench yesterday. Three -- Burger, Brennan and Marshall -- were traveling. Powell was in the Mayo Clinic in Rochester, Minn., for a "routine checkup," according to the court's press officer.