The Nuclear Regulatory Commission cannot consider cost in setting and enforcing general safety standards for nuclear facilities, the U.S. Court of Appeals held yesterday, overturning 1985 NRC procedures for determining plants needing improvements to meet new standards.

The decision, considered a victory for nuclear safety groups, is the second major defeat in a week for the Reagan administration's regulatory reform program, which required cost-benefit analyses to be a part of the rulemaking on many safety questions.

In a unanimous decision last week, the full D.C. Circuit court ruled that the Environmental Protection Agency can consider only health, not cost or technological feasibility, in setting permissible emissions standards for toxic substances.

"We hold that the {NRC} may not take economic costs into account in fulfilling its statutory mandate to ensure adequate protection of the public health and safety," Judge Abner J. Mikva said yesterday, writing for the three-judge panel. He was joined by Judge Harry T. Edwards, and Judge Stephen F. Williams concurred. The decision came in a challenge to a new NRC rule on backfitting, or retrofitting, brought by the Union of Concerned Scientists. Any change in a nuclear plant after approval of the facility's construction permit is considered backfitting or retrofitting.

The rule allowed the commission to use cost-benefit analyses in setting any new general safety rules and in determining whether operating nuclear facilities would have to be updated, or retrofitted.

At the same time, the rule stated that cost should not be considered in providing "adequate protection." The court said the rule was confusing and described it as an "exemplar of ambiguity and vagueness; indeed, we suspect that the commission designed the rule to achieve this very result."

"The commission must determine the content of the adequate-protection standard without reference to economic costs; the commission must then apply that standard to individual applicants and licensees notwithstanding any pleas of poverty," Mikva said.

Costs can only be taken into account in deciding whether to require additional protection, above and beyond the statute's "adequate-protection" level, the court said.

Ellyn R. Weiss, an attorney for the scientists' group, said, "This is another example of the court holding the line against some of the excesses of this administration, in particular, the efforts to weaken the standards of protecting the public in the process of lowering costs of the nuclear industry."

William H. Briggs Jr., NRC solicitor, said the commission believed the decision was narrowly drawn, but basically upheld the process the commission had established on retrofitting.

"I will be surprised if the decision has any Draconian impact on anything," Briggs said.

Weiss said it was unclear how many decisions on retrofitting the NRC made using the new rule because the regulations for implementing it required the cost-benefit analyses to be performed early in the process. "We don't know how many were tossed out before they ever reached the point of formal consideration," Weiss said.

The court's decision may prohibit the NRC from approving a proposal to change its emergency planning ruling that is also based on a cost-benefit rationale, Weiss said. The proposal would exempt the Shoreham nuclear facility on Long Island and the Seabrook plant in New Hampshire from the requirement that states have emergency plans in effect before plants open.