The proposed Hampton Roads Energy Co. refinery in Portsmouth, Va., a troubled project in the heartland of the James River oyster and the Chesapeake Bay blue crab, appears headed for further setbacks in one of the nations's classic environmental battles.

Despite highly sophisticated operating procedures that are projected to be the cleanest that modern technology can produce, the proposed refinery lies bedeviled by the potential for some doomsday oil splill from tankers en route to or from its pipes.

"What it all boils down to is risk," said Sue Wilburn, director of the Virginia Council on Environmental Quality, "a value judgment on the level of risk you are willing to accept. And there is a level of risk in almost everything, whether you're building a nuclear generating plant of walking across the street."

Wilburn's 10-member council now is coordinating the state's final review of an Enviromental Impact Statement on the refinery - a review to be presented to Gov. Mills E. Godwin on Thursday.

Refinery proponents, which include the job-hungry city of Portsmouth aid state industrial leaders, say the state's support is absolutely crucial to its approval by the single regulating agency yet to be heard from - the U.S. Army Corps of Engineers.

Early state enthuiasm for the project appears to have cooled somewhat in the face of critical reports from three of the 31 U.S. agencies asked by the corps to comment on the environmental impact statement.

Gov-elect John N. Dalton, who campaigned unequivocally for the refinery in his recent race for the governorship suddenly has turned cautious.

"I am certain the refinery people have to be concerned about the . . . federal environmental agencies . . . lining up pretty much against it," Dalton said last week. "I don't want to do anything environmentally unsound . . . At the time I came out for the refinery, it appeared that the state environmental agencies were giving it conditional approval."

Dalton ultimately may have little to say about the refinery. Godwin could forward the state's position himself to the Corps of Engineers before he steps down next month.

The proposal at issue would create a $600 million plant on the Elizabeth River in Hampton Roads to process 175,000 barrels of all day brought in by tankers from the Persian Gulf.

It would be the largest oil refinery on the East Coast and the first built there in nearly 20 years.

In the past three years the refinery has been granted all its necessary permits except one from the Army Engineers for a channel dredging operation. It was in the course of evaluating that proposal that the engineers had an environmental impact statement prepared and circulated. Official oppostion to the refinery then began to stiffen.

Last month the National Marine Fisheries Service drew attention to the dangers of the proposed dredging resuspending lethal heavy metals, pesticides and other pollutants that have settled out of the river water in the past and now lie buried in river bottom silt.

Taking note of "major" oil spills at the Amoco refinery in nearby Yorktown in February, 1976, and June, 1977, it cautioned:

"A careful reading of the oil spill record in Chesapeake Bay within the last 18 months can only lead one to the inescapable conclusion that the incidence of oil spills is increasing, and that construction of a refinery in the fertile vulnerable water of lower Chesapeake Bay constitutes an imminent peril to the biota and fishing industry."

Several weeks later the U.S. Fish and Wildlife Service wrote the engineers that "we believe the potential for serious impacts to the bay's crab fishery is high should oil spills occur . . ."

Harshest of all were the comments of the U.S. Environmental Protection Agency. It said the dredging proposal "will have the potential to cause significant adverse environmental impact to the already stressed water quality of the Hampton Roads area.

"Furthermore, the cumulative impacts of large and small oil spills resulting from refinery terminal operations would be deleterious to the quality of aquatic life . . .

"The proposed action would seriously impact the future environmental quality of the region and essentially mortgage the environmental interests of future generations. Therefore we recommend that the Corps of Enginners deny this permit."

Hampton Roads Energy Co. vice president Robert Porterfield stung by the EPA comments, retorted that the agencies letters "contain unfounded misstatements, conclusions and opinions that cannot be supported by facts on the record. They contain language best described as 'vague and uncertain.'"

He said the company, backed financially by the Atlanta-based Cox newspaper chain, would continue to push for approval of the refinery and would prepare a report noting that "EPA is inconsistent in relation to the Hampton Roads project and other programs and projects currently in progress and planned for the future."

Col. Newman A. Howard Jr., Norfolk District engineer for the Army Corps, said federal agencies have until Thursday to comment on the corps impact statement, but that virtually all of the major agencies have been heard from.

Howard said rejection by the state effectively would kill the refinery at the local level, and that full support from the governor's office is needed just to keep it alive.

Opposition from the federal agencies, he said, already has assured that he will have to refer the proposal to a higher level in the corps for action, even if Godwin gives the project his support.

Wilburn said many of the initial federal and state concerns about the refinery have been laid to rest by investigation, consultation and compromise. He said many others could have been headed off if the full environmental impact statement had been prepared before the state permits were sought and issued.

For example, initial EPA concern about the refinery adding more hydrocarbons to the already smoggy air over Hampton Roads has been partially ameliorated by state action removing other hydrocarbon sources from the area.

Both Wilburn and Frederick Spreyer, a consultant for the energy company, said the overriding concern among those leery of the project is the prospect of oil spills - spills too massive for even the containment and cleanup equipment proposed by the refinery, which the National Marine Fisheries Service describes as "excellent."

"Ultimately that is the big risk we're all really talking about," said Wilbur. "We can argue about the size of it and debate probabilities but ultimately it is a value judgment on the level of risk we're willing to accept."