Former North Carolina governor James B. Hunt, still battered from his beating by Sen. Jesse Helms (R-N.C.) last year, took himself out of contention yesterday for his state's 1986 Senate race.

With Republican Sen. John P. East's second-term candidacy shadowed by uncertainty about his ill health, an open battle is in prospect for the key Senate race.

Hunt, the state's top Democrat for the last decade, told supporters that he believes that he could win "a close and hard-fought campaign" against East or any other Republican, but that financial and political burdens would require him to start "right away."

Instead of leaving his law practice and starting another race, he said, "I really believe it's time I put my family first."

Democratic officials here said they regret Hunt's decision, and Republicans said it improves their chances of retaining the seat. In North Carolina, views were mixed as to whether a different Democrat might have a better chance.

Gary Pearce, a longtime Hunt aide, confirmed that he and others who helped Hunt in two successful gubernatorial campaigns and his close but unsuccessful fight with Helms, had urged him to step aside.

Other sources said polls showed Hunt carrying many scars from his battle with Helms, in which the incumbent spent a record $16 million, much of it on negative television advertising attacking Hunt.

"There clearly were some negatives left from last year," Pearce said, "and you had to assume there would be the same attacks again. The problems were not insurmountable, and the governor thought he could raise $5 million for the race, which is probably what he would have needed. But he didn't want to put his family and his friends through the same thing again."

Former Democratic governor Terry Sanford, who retired recently as president of Duke University, was first to move after Hunt's announcement, saying he will become a candidate Oct. 1. But several top Democrats indicated that Sanford, last elected in 1960, will face opposition from "fresh faces."

William C. Friday, president of the University of North Carolina system, is the choice of several influential Tar Heel Democrats. He has never run for office but has a built-in network of potential workers among alumni and supporters of the university.

Speculation about others who may take a look at the race centered on D.M. (Lauch) Faircloth, who sought the gubernatorial nomination unsuccessfully in 1984 and has the personal finances to make another race; Wade Smith, current Democratic state chairman; J. Phil Carlton, Hunt's law partner, and possibly Rep. Charlie Rose (D-N.C.) and state Sen. Marshall Rauch (D).

East, who won an upset victory in 1980 with help from Helms' organization, the Congressional Club, has been absent from the Senate much of the year while receiving treatment for what has been described as a thyroid illness. He remained home in North Carolina when the Senate returned this week.

Tom Fetzer, a spokesman for the Congressional Club, said: "We are still hoping Sen. East will recover and be able to run but, if he is not, we will have a candidate ready."

That is expected to be Norman Wiggins, president of Campbell College and head of the state Baptist convention, who said this week he would like to run if East did not.

Other possible Republican contenders include state Senate Minority Leader William Redman Jr. and former Charlotte mayor Kenneth Harris.