In the early 1970s, John F. Kerry, a dashing former naval officer, led the Vietnam Veterans Against the War in nationwide protests, capturing headlines with his ringing challenge to the Senate Foreign Relations Committee: "How do you ask a man to be the last man to die for a mistake?"

In the mid-1970s, Raymond Shamie, a self-made millionaire who founded a lucrative high-technology company, joined the John Birch Society and founded a program called "Wake Up! America" to school his 500 employes in conservative economics and warn them of a communist threat.

Today, as the two men face each other in one of the most vitriolic Senate races in the nation, their controversial pasts have become the focus of the campaign.

"Its like entering a time warp," said Shamie, fending off what he calls the Kerry camp's "mudslinging character assassination and Joe McCarthy tactics" in criticizing his past ties to the Birch society.

Kerry, accused by Shamie of favoring "unilateral disarmament" and attacked by a Shamie associate as "near-treasonous" for his antiwar record, decries the GOP's "redbaiting and scare tactics."

"They're trying to lefty me," he said.

To a great extent, the contest between Kerry, 40, the liberal Democratic lieutenant governor, and Shamie, 63, a political novice who defeated Elliot L. Richardson in the GOP primary, is tied to the battle between President Reagan and Democratic presidential nominee Walter F. Mondale in this state. Shamie, with his folksy demeanor and unabashed conservatism, is clutching Reagan's coattails. Democrats fear that a poor showing by Mondale could sink Kerry.

The National Republican Senatorial Committee calls the race the GOP's best chance to capture a Democratic seat among the 14 being contested this year. Sen. Paul E. Tsongas (D-Mass.) announced in January that he had cancer and would retire to spend more time with his family.

For Massachusetts, the ideological choice reflects a division more than a decade in the making. This was the only state George McGovern carried against Richard M. Nixon in 1972. Democrats still outnumber Republicans here almost 4 to 1.

But the tax-cutting fervor that swept the nation in the late 1970s hit Massachusetts in the form of Proposition 2 1/2. Conservative Democrat Edward King had ousted Gov. Michael S. Dukakis, a liberal, in 1978, although a more moderate Dukakis was reelected in a 1982 rematch.

In recent years, the state's economy has boomed, and its high-technology industries are filled with independent voters far removed from the concerns of traditional Democratic constituencies. In 1980, Reagan carried the state by a bare plurality over President Jimmy Carter and independent John B. Anderson.

Last week, in the wake of publicity over Shamie's Birch society ties, a WNEV-TV survey found Kerry leading by 9 percentage points, thanks to a shift among independent voters -- 40 percent of the state's electorate -- toward the Democrat.

Nonetheless, Democrats are worried that a Reagan victory, coupled with Shamie's financial resources, could give Shamie the edge on Nov. 6. With a personal fortune estimated at $20 million, Shamie spent $1.3 million running against Sen. Edward M. Kennedy (D-Mass.) in 1982 and nearly $1 million against Richardson in this year's primary.

The campaign began last month with a clash over taxes as Shamie repeated his pledge never to vote to raise them and attacked Kerry for saying during the primary that "we've been left no choice but to accept some kind of tax increase to get us out of a major crisis" of billion-dollar deficits.

Since then, Kerry has pledged not to vote to raise taxes but instead to cut expensive weapons programs such as the MX missile and B1 bomber and to slash agricultural subsidies.

Over the last two weeks, however, Shamie has found himself on the defensive. A memo he wrote in 1975, inviting nine friends and businessmen to a "political action meeting" with a Birch society official, was leaked to the press. Shamie wrote that the meeting "will be very helpful in understanding and evaluating our country's movement toward socialization . . . . Enclosed is a copy of 'The Naked Capitalist,' a book review describing the conspiracy in some detail."

Shamie acknowledged that he had had "a flirtation" with the Birch society but said he had dropped out because he had found it too extreme. "These are not violent people," he said. "They are very sincere, patriotic Americans. They are off on a tangent, sure, but they're certainly nothing to be ashamed of or frightened about."

In another leaked memo, Shamie's former campaign manager wrote that the library of Shamie's company, Metal Bellows Corp., which had stocked material from the Birch society and the Liberty Lobby, another right-wing group, had been "purged of some of its more controversial elements" in anticipation of the campaign.

Last week, Kerry attacked Shamie for "embracing" the Birch society. "It is imperative that all of us who care about the survival of our country rise up and name this political filth for what it is: we should denounce its racism, its anti-Semitism, its sexism, its profound anti-Americanism," he said.

Kerry also seized on another leaked memo, dated September 1979, in which Shamie solicited businessmen's contributions to lobby against affirmative-action rules for women and minorities, contending that they amounted to "legalized discrimination."

Shamie now says he supports affirmative action and opposed only racial and sexual quotas.

Meanwhile, Shamie supporters counterattacked. "It's time for John Kerry to be scrutinized on his past," Republican State Committee Chairman Andres S. Natsios said. Natsios questioned why Kerry had spoken in 1971 to the National Peace Action Coalition, allegedly a Trotskyite offshoot of the Socialist Workers Party.

Natsios also accused Kerry of "duplicity" in tossing away another veteran's medals, rather than his own, in a dramatic 1971 protest on the steps of the U.S. Capitol. Kerry, who proudly displays his Silver Star and other decorations in his apartment, acknowledged the incident, saying he had disagreed with the decision to throw away medals but agreed to toss those of another veteran at the man's request.

In a news conference, Birch society spokesman John W. McManus charged that "the Kerry record includes a heavy association with communists," citing a banner of Communist Angela Davis held aloft during a VVAW antiwar protest.

Joining McManus, retired general George S. Patton, son of the famous World War II general and honorary chairman of Shamie's veterans' committee, called Kerry "soft on communism" and said that, by protesting the war, Kerry "gave aid and comfort to the enemy and probably caused some of my guys to get killed . . . . There's no soap ever been invented that can wash that blood off of his hands."

Furious, Kerry called a news conference to declare, "I have served my country and I have fought for my country. I have bled for my country and I love this country." Patton and the Birch society, he said, "slandered me for my work in trying to stop the war."

Shamie, saying he had known nothing about the Birch society news conference, disassociated himself from the remarks and accepted Patton's resignation from his campaign post.

However, Shamie's criticism of Kerry's stands on defense last week seemed designed to raise doubts about the Democrat's attitude toward the Soviet Union. "John Kerry would give a freeway to the Soviets to send their missiles over here," he said, criticizing Kerry's opposition to an anti-missile space weapons program.

The Mid-America Conservative Political Action Committee said this week that, despite opposition from Shamie, it would run radio ads and distribute 250,000 leaflets "exposing Kerry's pro-abortion, pro-homosexual stands" and contending that his protests aided the Vietcong.

Beyond the increasingly strident attacks, broader perceptions may determine the outcome of the race. In the fishing town of Gloucester, State Rep. Pat Fieri, a Kerry supporter, said that Shamie, a truck driver's son, "appeals to blue-collar workers because here's a guy who's gotten a piece of the rock with his bare hands. They don't want higher taxes and neither do the young professionals. It's the idea of 'I've got mine, buddy.' "

But Kerry, a Yale graduate and the son of a diplomat, impressed Gloucester's ethnic residents last summer by giving a fiesta speech in Italian. Last week, longshoreman Arnold Carrico recalled Kerry's antiwar activity with approval and dismissed Shamie with the remark, "We've got enough millionaires in Washington already."