A House Armed Services panel yesterday refused to authorize the Energy Department to begin constructing a $571 million factory to process plutonium for nuclear weapons at its Rocky Flats, Colo., weapons plant.

The Defense Nuclear Facilities Panel, chaired by Rep. John M. Spratt Jr. (D-S.C.), rejected Energy Secretary James D. Watkins's request for $65 million to begin work on the project in fiscal 1991. The Senate Armed Services Committee had previously taken similar action.

The two votes represent the most serious rebuff Congress has delivered to Watkins in his attempts to revitalize DOE's 12-state network of nuclear weapons production facilities. It is apparenty the first time Congress has publicly turned down funds requested for a major nuclear weapons facility since the Manhattan Project.

Watkins had told Congress that the new facility, known as the Plutonium Recovery Modification Project (PRMP) is "essential" and "urgently needed" not only to produce pure plutonium required for warhead detonator assemblies but also to improve worker safety at Rocky Flats and to process thousands of cubic yards of radioactive wastes that have accumulated there.

But Spratt's panel said Watkins has not proved the facility is needed because he has not submitted a five-year plan for modernizing the weapons complex that Congress ordered.

Plutonium, a man-made radioactive metal that can be lethal if inhaled, is the main ingredient of the "pits," or detonators, manufactured at Rocky Flats. The United States has no other such facility.

Manufacturing operations at Rocky Flats were suspended in December for safety reasons and could resume without construction of PRMP, but officials say the plutonium supply is running low. The United States is not producing any new plutonium and Watkins has described PRMP as essential to safe, efficient recycling of plutonium from existing warheads.

Members of Colorado's congressional delegation have joined environmental groups in demanding that Rocky Flats, 16 miles from Denver and 11 from Boulder, be closed and operations moved to a more remote site. Watkins has agreed with them in principle, but said that such a transfer will take many years and that PRMP is needed in the meantime. Spratt's panel rejected his position.

"The panel recommends {to the full Armed Services Committee} that the Department of Energy expedite its plans to transfer the plutonium processing mission to a new site" by beginning "immediately" to search for a new location, it said after marking up part of the 1991 defense authorization bill. The panel asked for an environmental impact statement assessing alternatives to Rocky Flats and authorized the Energy Department to spend $15 million on a "non-site-specific design" for a new facility.

"A great victory," said Rep. Patricia Schroeder (D-Colo.), a member of the panel. "Maybe next year they'll have the five-year plan" to justify PRMP, she said, "but with the Cold War melted down, you don't have that overriding national security thing beating on you all the time." She predicted there would be no attempt to restore the construction funds when the authorization measure goes to the House floor.

Melinda Kassen, an Environmental Defense Fund lawyer in Boulder and strong critic of Rocky Flats, called the decision "an excellent sign that Congress is taking a serious look at the Department of Energy's wish list instead of just rubber-stamping it."

Energy Department spokeswoman M.J. Jameson said the PRMP decison was "not a tragedy." The environmental assessment of alternative sites "was something we were going to do anyway," she said, "and if it shows it shouldn't be built at Rocky Flats, we won't build it there."