A New Jersey antinuclear group charged yesterday that three utilities and the New Jersey state police have illegally harassed and spied on nuclear protesters in the state since 1977.

In a lawsuit filed in New Jersey Superior Court in Essex County, the Safe Energy Alternatives Alliance said Public Service Electric and Gas Co.; Jersey Central Power and Light Co.; and Atlantic City Electric Co. conducted illegal surveillance of nuclear protest demonstrations.

The antinuclear group also charged that state police in New Jersey broke their own guidelines against political surveillance.

"There's been a history in New Jersey over the years of police and private security guards monitoring public meetings and rallies," said Frank Askin, an attorney for the Constitutional Litigation Clinic at Rutgers University Law School, which is representing the antinuclear group.

"We're contending that this is an abridgment of free speech and our right to protest legally and we want it stopped," Askin said. The suit seeks court order against additional surveillance.

A New Jersey state police spokesman said the police have guidelines governing intelligence gathering, but the spokesman declined to comment on the suit.

According to the suit, state police took pictures of antinuclear demonstrators in 1978. It also charges that a local police detective turned away persons on their way to a panel discussion on nuclear power this year in Berkeley Township, N.J., and copied license plate numbers at the discussion.

The lawsuit also charged that from 1977 to this year, employes of the three utilities photographed antinuclear demonstrators and copied their license plate numbers during protests in New Jersey.

On one occasion last year, antinuclear group's complained alleged, two Jersey Central employes photographed license numbers and participants at a demonstration and told members of the antinuclear group that they worked for two state newspapers.

The complaint also charged that Atlantic City Electric hired a part-time investigator to infiltrate the antinuclear group after it protested the utility's plans to construct offshore nuclear power plants.

The antinuclear group said Jersey Central spent $595,437 on private security agencies in 1976, the Atlantic City utility spent $299,254, and Public Service spent $60,184. The suit does not specify how much of that was spent on surveillance as opposed to normal plant security.

A spokesman for Public Service Electric and Gas Co. said yesterday that the utility had not seen the law suit."I'm not aware of any action such as they allege," he said, "but I can't categorically deny it at this time." Spokesmen for the other two utilities could not be reached for comment.

An official of the Campaign for Political Rights, a group here that monitors police surveillance activities, said similar suits charging police spying on antinuclear activists have been filed recently in Buffalo and Los Angeles.