The Soviets weren't even at the bargaining table, but Glen Cove, N.Y., began to taste victory yesterday in its cold war with Moscow.

The mayor of Glen Cove announced -- after a session with a top man from the State Department -- that it may not be long before the Soviets are allowed back on the Long Island town's sandy beaches.

The State Department, the mayor happily reported, has agreed to back Glen Cove on this basic proposition: If the Soviets want to play, they'll have to pay.

Rep. John LeBoutillier (R-N.Y.), who arranged the get-together between Glen Cove Mayor Alan M. Parente and William Schneider Jr., undersecretary of state-designate for security assistance, was equally enthusiastic.

"Peace," LeBoutillier proclaimed, "is at hand." He said Schneider had also promised administration backing for a still-to-be drafted bill that would supply Glen Cove with federal funds in place of the property taxes the Soviets don't pay in the small community -- an estimated $100,000 a year.

The Soviets have occupied a mansion on a 37-acre estate in Glen Cove since the creation of the United Nations brought them to New York in 1946. According to Parente, the people of Glen Cove have been especially unhappy since 1966, when the State Department pressured the town into according the Soviets tax exempt status.

But the last straw came this spring when the Long Island newspaper Newsday reported that the Soviets had installed sophisticated eavesdropping and surveillance equipment on the top floors of the 49-room mansion.

Told by Washington, in effect, that nothing could be done, the City Council voted in May to ban the Soviets from its recreational facilities, including the beach, the municipal golf course, the fishing pier and the tennis courts. Soviet complaints, conveyed in a stern note from the State Department, did no good. The City Council voted 6 to 1 last month to stand fast.

The Soviets retaliated by banning American diplomats in Moscow from swimming in the Volga, but the mayor wasn't impressed. He described the spot that had been put off limits as "a beach six miles out of Moscow that looks like a mudhole."

Parente, however, said he was anxious to reach a settlement and thus avoid "an escalation in retaliation."

He declined to suggest just how much the Soviets should pay for the beaches and all. Actually they had been paying a bit already, like other Glen Cove residents, for the golf course ($15 a year plus greens fees) and for the tennis courts ($15 for a yearly permit). But the beaches are free for residents, and Parente said he feels the Soviets should be willing to accept an overall "increase in fees."

After all, he pointed out, American diplomats in Moscow are customarily socked $35 an hour for use of a Muscovite tennis court. Putative diplomatic softball teams, Parente added, were told by the Soviets several years ago that it would cost them $100 every time they wanted to use a softball field.

The State Department's Schneider was said to be busy trying to arrange a meeting with Soviet diplomats here and did not attend the news conference. But LeBoutillier said Schneider had promised support for a bill that would permit federal funds to be paid in lieu of property taxes to small residential communities with tax-exempt diplomatic missions in their midst.

LeBoutillier said this would apply to Glen Cove and to a number of other towns, such as Sparta, N.J., which has a Czech mission, but not to big cities such as Washington. "There's a difference between Glen Cove and Washington," he said.

LeBoutillier said Schneider also had pledged backing for a bill that would prescribe immediate deportation for diplomats whom the president "has reason to believe" are engaged in electronic surveillance and who fail to heed official demands that they desist.

So Glen Cove appears to have won?

"I think, most certainly, that we did," the mayor said.