The Supreme Court yesterday for the first time established constitutional rights for people committed to institutions for the mentally retarded, including unprecedented, but limited, guarantees of a minimum level of training.

The court also said institutions have an obligation, enforceable in the courts, to provide a reasonable amount of physical freedom as well as safety for involuntarily committed patients.

At the same time, Justice Lewis F. Powell Jr., writing for the 8-to-1 majority, cautioned judges to respect the professional judgments and budgetary restraints of the institutions and not require them to "make each decision in the shadow" of possible lawsuits.

The opinion, carefully balancing the competing interests in one of the most publicized cases of the current term, applies directly to hospitals for the mentally retarded but also is expected to affect mental hospitals. There are an estimated 150,000 people in state institutions for the mentally retarded.

"Persons who have been involuntarily committed are entitled to more considerate treatment and conditions of confinement than criminals whose conditions of confinement are designed to punish," Powell said.

The decision left many questions unanswered, such as what constitutes reasonable freedom, and many points unclear, in an apparent attempt to allow flexibility for professional judgments. It also allowed numerous defenses for hospital officials sued for mistreatment, including a defense that the problems were caused by "budgetary restraints."

Nevertheless, the decision is a cornerstone in what has become a "patients' rights" movement comparable in many respects to the prisoners' and defendants' rights thrust of the '60s and '70s. Nearly all the states face litigation in this field.

Twenty-one, fearing federal judges would soon begin looking excessively over their shoulders, asked the Supreme Court to resolve the issues, preserving to the extent possible maximum flexibility for their state institutions.

Yesterday's case began with a suit brought on behalf of Nicholas Romeo, 33, a man with the mental capacity of an 18-month-old child. Romeo's mother had him legally committed in May, 1974, to the Pennhurst State School and Hospital near Philadelphia. That state-run institution has been the subject of numerous suits and complaints of mistreatment.

She became concerned about her son's treatment after learning that he had been injured at least 70 times both by his own hand and by others reacting to his aggressive behavior. She also learned that officials had repeatedly confined him in physical arm restraints during portions of each day.

She sued the officials for damages under federal civil rights laws, seeking broad constitutional guarantees for patients to be free of physical restraints, and guaranteed rights to comprehensive training and development programs.

She lost at trial, but the 3rd U.S. Circuit Court of Appeals, in a divided ruling, ordered a new trial because of misinterpretations of constitutional law by the District Court.

Pennhurst appealed, but yesterday the Supreme Court agreed with the appeals court and issued its own guidelines for these cases. The justices agreed unanimously that the case should be returned to the lower court for a new trial.

Chief Justice Warren E. Burger agreed with much of the ruling, but dissented from any right to treatment.

Powell said that the patients should have at least the constitutional protections afforded prisoners, such as a right to safe conditions and the right to be free from unnecessary physical restraints. He noted that unlike prison inmates, the patients at issue in yesterday's case "may not be punished at all."

The right to treatment is a more difficult problem, Powell said, because the Constitution guarantees no substantive services to anyone. He said the least that could be demanded, however, was that the involuntarily committed receive the training they need to function safely in the hospital, without hurting themselves or others. This would also help them avoid the need for shackling and physical confinement.

Powell based his ruling on the due process clause of the 14th Amendment, which protects the personal physical liberty of individuals from unfair or unreasonable incursions by the states. The question, he said, was how to determine what is fair in this situation.

" . . . In determining what is reasonable," he said, "we emphasize that courts must show deference to the judgment exercised by a qualified professional. By so limiting judicial review of challenges to conditions in state institutions, interference by the federal judiciary with the internal operations of these institutions should be minimized. Moreover, there certainly is no reason to think judges or juries are better qualified than appropriate professionals in making such decisions."

The decisions of the professionals should be considered generally valid by the courts, Powell said, unless there is "such a substantial departure from accepted professional judgment, practice or standards as to demonstrate that the person responsible actually did not base the decision on such a judgment."

Experts in mental health law said yesterday that even with its ambiguities, the ruling in Youngberg v. Romeo was an important change in the law.

"It is a positive step in the right direction," said Norman S. Rosenberg, director of the Mental Health Law Project. Rosenberg said it was the first time the court had said that such institutions have to do anything besides basic maintenance for patients, the first time any "affirmative right" to training had been granted. Joel Klein, who filed a friend of the court brief for the American Psychiatric Association expressing concern about maintaining professional autonomy for hospital officials, said he felt that need had been satisfied.

The court also disposed yesterday of a related case. The court said an intervening decision of the Massachusetts Supreme Court required return of Mills v. Rogers for further consideration in the lower courts.