The Virginia State Board of Health today lifted most of a five-year ban on fishing in Kepone-contaminated sections of the James River, leaving only three species off limits because of persistently high levels of the suspected carcinogen.

The ban was imposed in late 1975 by then-Gov. Mills E. Godwin after an Environmental Protection Agency study found a major portion of the river and its marine life had been tainted by the pesticide. It was produced by two companies in Hopewell and secretly dumpted into the river over a period of years in what state officials called a major environmental disaster.

State health officials have said concentrations of the chemical found in many fish have declined in recent months and that the river should be reopened to commercial fishermen. The call to lift the ban, presented by Dr. Robert Stroube, assistant health commissioner, was passed unanimously in a matter of minutes. Watermen still are prohibited from catching striped bass, eel and croaker.

Stroube, saying Kepone levels in other James River fish were "rather good," partyly attributed the decrease to a "function of time." He said, "The river is obviously cleansing itself more and more as the years go by."

The fishing ban was partly lifted from Jan. 1 to June 30 because of declining Kepone levels. The board had considered closing the river during the summer because of fears Kepone levels could increase but noted that recent catches indicate levels in some species are below what federal regulations permit.