Although he was seen fleetingly last week on Main Street in Montpelier, Vt., exiled Soviet author Aleksandr Solzhenitsyn has retreated again into his self-imposed archipelago in the nearby rural town of Cavendish.

Inside a barbed-wire-topped fence surrounding his secluded 50-acre tract in South Central Vermont - accompanied by electric-eye alarm devices and closed-circuit television monitors - Solzhenitsyn presumably has found the solitude that evaded him after he was expelled from the Soviet Union upon winning the 1970 Nobel Prize for Literature.

But his peace may have been found at the expense of the local townspeople, who by tradition are more neighborly than Solzhenitsyn appears to be.

"It upsets the hell out of all of us. You know he's here, but you can never say you have actually seen him," Quentin Phelan, town manager of Cavendish (pop. 1,264), said yesterday.

Since Solzhenitsyn moved into his $150,000 rural estate last October and made $250,000 worth of renovations, nobody in Cavendish has actually laid eyes on him, although everybody says they know he is there.

In September, the property's owner-of-record, Russian-speakin architect Alexis Vonogradov, blandly denied to a reporter that Solzhenitsyn planned to move to Cavendish, even though the Soviet author had listed the town as his intended residence when he had applied for a visa.

Vinogradov is a friend of Solzhenitsyn's from the Soviet emigre community in Montreal, and he hinted to a Washington Post reporter who showed up at the front gate that his answer might have been less than truthful.

When pressed about his plans to turn the house over to Solzhenitsyn, Vinogradov said with a wry smile. "I have no plans to do that, but if I did, I would give you the same answer."

Since then, Solzhenitsyn - or anybody who admits to knowing him - has refused to take telephone calls to the Cavendish house.

Visitors to the property find no mistaking Solzhenitsyn's quest for seclusion. Besides the fence, there are numerous "No Trespassing" signs, the electric eyes, the television monitors and a device that accepts magnetized cards to signal the heavy gate to open.

Once the electric eye beam is broken, a speaker box blares sternly, "What do you want?" When the names of Solzhenitsyn or Vonogradov are mentioned, the voice - presumably of Soizhenitsyn's American bodyguard - answers, "There's nobody to talk to. You can't come in," and the speaker goes silent.

While the townspeople may wonder why a man who fled such oppressive security in the Soviet Union would resort to barbed wire fences and electric eyes, some acquaintances of Solzhenitsyn's said they are not surprised.

"He has to have privacy. A lot of people are interested in him, and they are not all friendly," said Loring E. Hart, president of nearby Norwich University, where Solzhenitsyn lectured recently at a summer Russian studies program.

One Russian scholar conjectured, "Remember what happened to Trotsky in Mexico City."

Lloyd Stillwell, chief of Cacendish's three-man police force, said, "I don't know. Maybe the KGB would like to put an end to him." When he left Zurich, Switzerland, his first home after exile, last summer, Solzhenitsyn complained to friends that he had been harassed by the Soviet secret police.

Outside of being spoted momentarily last April in Cavendish's general store, the celebrated Soviet dissident had not been seen again until Elizabeth Slater, a reporter for the Barry Montpelier Times-Argus, did a double take Friday on one of Montpelier main streets.

After spotting Solzhenitsyn, Slater followed him to a parking lot, waited until the bodyguard had left and conducted a brief, impromptu interview, his first since he came to Vermont.

In the interview, Solzhenitsyn said he planned to start a nonprofit publishing business in Vermont that will distribute his works on Russian culture, history and religion.

He confirmed reports that he is living in Vermont with his wife and three children, but refused to disclose the exact site of his home, saying with a smile, "Just say I live in Vermont." He said he spends all his time writing.

After obtaining incorporation applications from the office of the Vermont secretary of state, Solzhenitsyn dropped by a local bookstore that carries his works, shopped nearby and had lunch in a restaurant before leaving.

"He wasn't very communicative," Slater said.

Hart said he planned to contact Solzhenitsyn through an intermediary to invite the author to lecture again at Norwich University's summer Russian Studies program. "My interest in him has been rekindled by his appearance in Montpelier. We hope to arrange something," Hart said.

Meanwhile, Phelan said, Cavendish's townspeople are anxiously awaiting the first friendly overture by their new neighbor.

"All he has to do is come out one time, say hello and tell us he likes his privacy. We respect that.The hell of it now is that we can't say we have ever really seen him," said Phelan.