If the public opinion polls were mistaken in some of last fall's election predictions, it may have been because the public did not cooperate with the polls and because the polls themselves are becoming too costly, a panel of experts suggested here today.

While it is not clear why several of lasy year's election polls were bably in error, the mistakes could have resulted from a decline in public response rates to the polls, the panel told the 145th national meeting of the American Association for the Advancement of Science. One survey by the University of Michign showed there had been a "modest decline" in response rates to polls -- enough to throw some of them off.

"There has been quite a lot of research looking for causes for the decline in the response rate, none of it conclusive," said Daniel Melnick of the Library of Congress. "The prevailing theory is that there has been a decline in public trust in all types of institutions, including polls."

The panelists, all experts in the "use and misuse of survey data." agreed that the skyrocketing cost of professional polling was forcing many organizations to cut corners and to rely more on untrained volunteers and telephone surveys.

"It may be an economic problem," said Philip E. Meyer, director of news research for the Knight-Ridder newspapers. "The cost of getting a response rate is going up and that's why there's been a shift toward telephone interviews."

The two biggest mistakes in last year's election polls cited by the experts were those showing incumbent Sen. Dick Clark (D-Iowa) and Thomas McIntyre (D-N.H.) far ahead of their challengers. Both Clark and McIntyre were defeated.

A poll conducted in mid-October by Cambridge Survey Research Inc. found that McIntyre was leading Republican Gordon Humphrey by 59.5 percent to 30 percent. Humphrey won with 51 percent of the vote.

A poll in early October by Peter D. Hart Associates found Clark leading his opponent, Republican Roger Jepsen, by 57 to 27 percent. Jepsen beat Clark with 51 percent of the vote.

"The survey organization [in the New Hampshire poll] said the poll was conducted by volunteers, not professionals," said Barbara Bailar of the U.S. Census Bureau. "But Cambridge Survey Research selected the sample and trained the volunteers. When things go wrong, interviewers are often blamed."

Bailar said the growth in the number of polls may be to blame in part for their inaccurate results. She said that survey spending for 1978 has been estimated at $4 billion.

"That is probably an underestimate." Bailar said, "since it did not include private surveys commissioned by politicians and, in an election year, there were many of these."

The panel said that while last year may have seen some dramatic polling mistakes, it also showed that, on the whole, polls were as accurate as any that had been taken in the past.

Melnick said Public Opinion Magazine analyzed polls taken in 17 states where governors were up for election. He said 15 polls chose the winning candidate.

Meyer said his organization conducted five preelection polls. He said all had picked the winning candidates.

"One reason may be that we conducted the polls just before the elections," Meyer said. "What's responded to in September does not always reflect voting intention."