The people in my town went to the polls angry last week. I hadn't seen anything like it in years, and I can't quite shake the scene.

Even today, it resounds back from my consciousness, back 20-odd years, to a time when I was on a 13-year-old poll worker for my father. I still remember the crusty man who hissed at a child, "I wouldn't vote for him if he were the last man on earth."

Tuesday, in the same town, in the same altered congressional district, I heard the echoes of that anger, and heard a renewed religious hostility. And it wasn't a comforting thing.

Last spring the pope had taken away my congressman, Bob Drinan. This fall, in an open letter days before the Democratic primary, Cardinal Medeiros warned Catholics that "those who make abortions possible by law . . . cannot separate themselves from that guilt which accompanies this horrendous crime and deadly sin."

Suddenly, going to the polls was an occasion for moral peril. Suddenly, the cardinal was a one-issue voter. With barely disguised glee, Howard Phillips of the Conservative Caucus announced that Medeiros had joined the "Moral Majority."

The cardinal did everything but name the names of the tainted candidates. But then, he didn't have to. The "sinners" would be the supporters of James Shannon, a Catholic, in the fifth district, and Barney Frank, a Jew, in my own fourth district. Both are for Medicaid funding of abortions. j

Well, the cardinal lost this election. Both Shannon and Frank were nominated. Some say "the letter" narrowed the margin of victory; others say it backfired. But no one disputes its effect on raising the level of anger.

The controversy brought 30,000 more voters to the polls than expected in my district. It brought Jews and Catholics angry at each other and/or angry at the church. It brought Democrats angry about abortion and Democrats angry about church interference. It brought cries of "right and wrong."

I think about it now because the town atmosphere is still rank with all sorts of unleashed bigotry. I think about it because what happened here is happening in other places.

Political platforms are now being scrutinized under lenses labeled "sin" and "morality." Politicians are telling us that God is on their side, as if God were a heeler. And the churches are becoming the new Tammany Hall of the 1980s.

Morality is a tricky label on any ballot. What is "morality," when the cardinal ends up supporting a man whose ads whispered, "focused busing . . . racial pressure . . ."?

Who is "moral" when Florida's Rep. James Kelly is give a 100 percent pure rating by the evangelical Moral Majority and is then captured on videotape by the Abscam crew? What is happening when the Christian Voice judges a legislator "moral" if he voted against sanctions to Rhodesia? When media minister Jerry Falwell equates godliness with the free enterprise system?

Yes, we all vote according to our sense of right and wrong. No, religious leaders don't give up their right of free speech when they put on the cloth or collar or robe. But it is unsettling to see people led to the polls in pursuit of Truth and Goodness by people whose Ten Commandments include the old militaristic conservative agenda. It is unsettling to see the anti-abortion movement manipulated into a fund-raising, envelope-licking ladies' auxiliary to the New Right.

It is even more unsettling to see that underneath the Politics of Morality lurks that oldest of evils, religious strife.

This is not, whatever we like to think, a country with a rich history of religious tolerance. Since 1628, when Thomas Morton was arrested in Massachusetts for the sin of dancing, we have needed protection from each others beliefs. We didn't separate what Jefferson called "this loathsome combination of church and state" out of tolerance but out of mutual fear.

Now this election, this whole political year, reminds me how easy it is for a self-proclaimed Moral Majority to label anyone they disagree with as an Immoral Minority.