Environmental groups, tired of being told they are more propaganda than power in the Reagan era, ran a major effort in the New Jersey election and think they won some political credibility there, even though their candidate apparently lost.

They say wait until next year, when "the green vote" will really flex some muscle nationwide.

Some conservatives on their target list, however, worry that what is coming might be more a "yellow vote," drummed up by ecopoliticians slinging emotional labels on issues too complicated for voters to fully understand.

Rep. James J. Florio (D-N.J.) appeared to have lost his run for governor by a narrow margin, following a campaign in which his environmental record was his No. 2 sales pitch after the state of the economy.

His Republican opponent, former state assembly speaker Thomas H. Kean, did a me-too routine on the importance of chemical control in this highly industrialized area, but the ecopoliticians say it was their support for Florio that brought him within a handful of votes to victory.

The New Jersey campaign pattern will repeat itself nationwide next year as environmentalists try to convert recent polls showing wide support for environmental law into actual votes, according to Bob Chlopak, head of a group called the Environmental Voters Alliance.

"We used New Jersey as an opportunity to work out models, develop skills, try things out, execute a plan and learn from it," he said. "Everything worked just fine, and we're very prepared to go into 10 or 12, maybe 15 states next year."

The EVA is a political action committee for a coalition of the Sierra Club, Environmental Action, Friends of the Earth, Solar Lobby and the League of Conservation Voters. In previous years, only the League got directly involved in electioneering while the others stuck to lobbying, leafleting, research and, yes, propaganda.

"That's going to change," Chlopak said. The groups raised about $40,000 for the New Jersey effort with special exhortations to their regular mailing lists. Chlopak said the response was the best the groups had had in years.

The money went into regular grass-roots campaign work, telephone banks, canvassing and get-out-the-vote preparations, all pounding home Florio's record in sponsoring the $1.6 billion "Superfund" toxic waste cleanup program. Calling himself "a flaming moderate," Florio worked hard to avoid the label of environmental extremism that has tarnished the cause in conservative eyes.

Even so, a campaign spokesman said the issue only brought in "a very small proportion" of Florio's campaign contributions, and that he spent 80 percent of his time on other issues, chiefly the economy. Chlopak said, however, that four state senate candidates his group helped ran well ahead of Florio.

"We were responsible for several thousand votes up here" in Bergen County, which the main Florio campaign largely neglected, Chlopak said.

EVA's New Jersey work and its fund raising are chicken feed compared to the media steak diet that congressional races will require next year. But Chlopak said the coalition won with small bankrolls in two state senate races and in the four races it tackled before Florio's: three New Jersey legislative primaries and the 1980 victory of Rep. Bob Edgar (D-Pa.).

Edgar won a fourth term by 11,000 votes in a district with a 3-to-1 Republican registration advantage, one where Ronald Reagan had a 45,000-vote margin. "The coalition had a tremendous impact, put out incredible effort," said Edgar's former deputy campaign chief, Will Robinson. Environment, it turned out, was an issue that cut across party lines.

"They could go in and talk to the Republicans where we couldn't," Robinson said. Environmental groups are clearly hoping to repeat that pattern next year.

Among the Republicans already feeling environmentalist heat is Rep. Don Ritter (R-Pa.). When he announced support last spring for a Clean Air Act package of amendments offered by Rep. James T. Broyhill (R-N.C.), "there was a great explosion in the Lehigh Valley," Ritter's district, a staff aide said.

A cascade of letters arrived at Ritter's Washington office, all accusing him of wanting to gut the Clean Air Act in the language used by the National Clean Air Coalition, the aide related. "They didn't show a deep understanding of the issues," he said.

Public meetings were "a stacked deck" against Ritter for a while thereafter, and press coverage followed suit, rarely doing more than mouthing labels, the aide said.

This is the conservatives' nightmare and the environmentalists' unmentioned weapon: the issues are so complex that voters might listen more to simplistic labels of what a position means.

Favoring extension of a deadline for compliance with an emission standard, or preferring "new source performance standards" to "lowest achievable emissions rate," does not necessarily mean a politician wants to gut the law, Ritter's aide complained. But rebutting such charges requires so much detailed explanation that it looks like waffling.

"Sure, there's that fear there among every one of the cosponsors of the Broyhill bill," the aide said.

Interior Secretary James G. Watt has become a major fund raiser for both the Republican Party and, in absentia, for environmental groups who see him as the enemy incarnate. But conservatives argue that polls showing 80 percent of the voters defending wilderness areas, the Clean Air Act and the like are warped by this simplistic labeling problem.

"Who's going to say they favor dirty air?" said an Environmental Protection Agency official. "It all depends on how the question gets asked."

Pollster Lou Harris told an Oct. 15 congressional hearing that "not a single major segment of the public wants the environmental laws made less strict."

He said his questions did not use simplistic labels but asked specifically whether people favored "pollution standards to allow power plants to burn higher sulfur content oil and coal," or "postponing current deadlines" for meeting pollution standards, or "relaxing pollution standards affecting human health if the costs are too high."

In every case the public said a resounding no, Harris said. "The message and the deep desire on the part of the American people to battle pollution is one of the most overwhelming and clearest we have ever recorded in our 25 years of surveying public opinion," he continued. "You mess around with the Clean Air and Clean Water acts and you are going to get into the deepest kind of trouble."

He added that the trend had turned upward recently, particularly in the West, possibly because more people feel that controls on pollution and development are being threatened. Environmental groups clearly expect to translate that feeling into votes.

But Rep. Cleve Benedict (R-W.Va.) produced results at that hearing from another Lou Harris survey done for corporate clients that found 94 percent of the public saying "costs should be considered when the government sets air, water or noise pollution standards." The question didn't mention health effects.

"What we are measuring constantly are not rational opinions based upon a study of the facts," Harris acknowledged. "These are people's gut reactions . . . to concepts, to perceptions. And this is the way we run our system."

That truth was acknowledged in passing by Sen. Malcolm Wallop (R-Wyo.), a strong supporter of further western energy development, when he advised 1,000 oil industry people meeting in Denver this month that they should stop pushing for permits to drill in wilderness areas.

"I am convinced you do not need image-tarnishing challenges over oil and gas development in wilderness areas to add to your PR problems at this moment in time," he said. There is "a brewing and unmistakable undercurrent of concern" about the rate of western states' development, he went on. "You cannot avoid this rising tide."

The New Jersey results will be closely analyzed to see whether the tide washed as far as the voting booth, and for hints as to whether it will still be high next year.

"Next year we'll have a track record," the EPA official said. "The Reagan Clean Air Act isn't going to be a monster and people will move on to worry about something else. These folks better be careful they don't crow too soon."