Harry Kelley, the mayor of this popular resort town, led a stream of opponents testifying today against designating a site 140 miles off the coast here for at-sea burning of toxic wastes, including PCBs.

Kelley won spirited applause when he told a panel from the Environmental Protection Agency, which proposed the site, "we have no responsibility to New York, New Jersey or Philadelphia to take care of their waste. This is not a threat, it's a vow: I'll fight you."

EPA wants to set aside a 30- by 40-mile rectangle of ocean due east of the Delaware-Maryland coast, where incinerator ships would burn toxic wastes at 1,200 degrees centigrade. The site would be the second approved, joining an area in use since 1976 in the Gulf of Mexico.

But Kelley and U. S. Rep. Roy Dyson (D-Md.), along with aides of Sen. Paul Sarbanes (D-Md.), Sen. Joseph Biden (D-Del.) and Rep. Tom Carper (D-Del.) and private citizens and environmentalists, questioned EPA's ability to monitor burns to assure they met standards. Dyson questioned the beleaguered federal agency's integrity and credibility in light of recent accusations that toxic waste "Superfund" money was used for political leverage.

But EPA appears wedded to the plan. Three officials at the hearing in City Hall defended the efficiency of the at-sea incineration process and the suitability of the location. They stressed that site designation is only one step toward an actual burn, and that any applicant would have to pass a separate hearing process before a permit was issued.

Alan Rubin, chief of EPA's water quality criteria section, said tests conducted at a PCB burn in the Gulf found no toxic wastes emitted from the stacks of Vulcanus I, the only incineration ship now licensed in the U. S. He said the principal emission was hydrochloric acid, which he said was almost immediately neutralized in sea water, and carbon dioxide which dissipated in the air.

"Our conclusion," said Rubin, "was that there was no detectable impact" from burning PCBs or other organic toxic wastes. A small residue of ash left after a burn is disposed of on shore in licensed toxic waste dumps, he said, and there is no dumping at sea from the process.

Opponents, protesting that the proposal imperils the area's most precious resources, the ocean and beach, called EPA's scientific data incomplete. Joyce Rosenthal of Greenpeace cited other studies in which traces of PCBs were found in incineration plumes. She added, "With the track record of Chemical Waste Management which owns Vulcanus I , you can understand the concern of the public."

In February, CWM's representative, James W. Sanderson, was accused by EPA sources of using his influence with former EPA Administrator Anne Burford to speed a request for a burning permit for Vulcanus I. Sanderson was a paid adviser to Burford at the same time he was on the CWM payroll, according to the accusations.

The company also was accused in EPA documents of keeping two sets of books on a Denver landfill it operated, which could have concealed from investigators a chemical leak at the dump site.

Biden, in a prepared statement, said recent controversies "don't inspire confidence" in EPA. He also questioned EPA's thoroughness in looking at the dangers of "major spills and accidents."

Only two parties testified in favor of ocean incineration--a spokesman for At-Sea Incineration Inc., which is building two incinerator ships, and a private citizen, Daniel Royle of Bayard, Del., who maintained that as long as there are toxic wastes, they have to go somewhere.

That argument buoyed Steven Schatzow, who ran the hearing for EPA. "There are very few good, environmentally sound ways of getting rid of toxic wastes," he said. To EPA, he said, at-sea incineration is one of them.