One month after five Soviet political prisoners were exchanged with the United States for two Soviet spies, most of the prisoners' families who also were guaranteed passage to the West are still locked in a struggle over the terms of their departure.

Soviet visa officials have refused to discuss some of their requests and have spurned efforts by U.S. diplomats on their behalf. Secret police follow and photograph them and telephone calls from the freed men to their families have been blocked or gone dead in midsentence.

The swap of Alexander Ginzburg and four other dissidents for the spies was hailed as a positive gesture by the Kremlin toward improved relations with the United States on the eve of a Senate debate on the new strategic arms limitation treaty to be signed next month in Vienna. But the continuing troubles faced by the families could sour the presigning atmosphere.

The exchange April 27 of Rudolph Chernyayev and Valdik Enger for Ginzburg, Mark Dymshits, Eduard Kuznetsov, Valentin Moroz and Georgi Vins included the men's "immediate families." This apparently is where the principal difficulty lies.

The Ginzburgs consider their immediate family to include his aged mother; his wife, Arina; their two young sons; and a 20-year-old neglected youth named Sergei Shibayev they took in several years ago.

Soviet emigration officials have crossed Shibayev's name off the list submitted by Ginzburg from the United States and have refused to discuss his status with Mrs. Ginzburg or, apparently, with U.S. Embassy officials.

The youth, who was drafted into the Soviet Army although he has a severe leg deformity from a childhood disease, faces reprisals from his officers for seeking to emigrate with the family, Mrs. Ginzburg said today in an interview in her apartment outside central Moscow.

"I cannot leave without him," she declared. "He may die without us."

She said the youth is in frail health. Dissidents here generally believe he was drafted as a reprisal against the Ginzburgs, who have been leaders in the Moscow dissident community. The KGB secret police sought unsuccessfully last year to force Shibayev to testify against Ginzburg at his trial for anti-Soviet slander.

Mrs. Ginzburg quoted one senior Soviet emigration official named Gardeev as telling her, "I think this question will be solved, but not the way you want it. Shibayev will be in the Army and serve his time."

She said the authorities have twisted the "immediate family" agreement in the case of Vins' aged mother, who is in poor health and elected not to go to the United States to join her son, saying she wished "to die in the motherland." The Soviets said unless the elderly woman left, they would not allow any other family members to leave.

This includes Vins' wife and five children. Vins, the pastor of an unofficial Soviet Baptist group represented by the government, became acquainted with President Carter, who is also a Baptist. The elder Mrs. Vins was compelled to agree to leave the country, Mrs. Ginzburg reported.

At the same time, Mrs. Ginzburg said, the Soviet have refused to allow the family to include a niece of Vins who desperately wants to join her uncle and aunt in America.

Of the other three freed prisoners, Dymshits' family already is in Israel and Kuznetzov's elderly mother has decided to leave the Soviet Union despite a severe nervous condition. Moroz' family is so far having no success in trying to pry the historian's writings in camp away from the authorities.

It took Ginzburg until May 9 to get a telephone call through after dozens of efforts, Mrs. Ginzburg said. May 9 is a Soviet holiday marking the defeat of Nazi Germany and Mrs. Ginzburg suggested with a wry smile that the secret police who listen to international calls were off duty.

Successive calls were blocked until May 18, she said, when Ginzburg by prearrangement called the U.S. Embassy here and talked with his wife for 40 minutes. "It was wonderful," she said.

But another call to the embassy on May 23 was broken after a few minutes and she is convinced the Soviets did it.

Embassy officials reached tonight refused to discuss any of these matters. Routine international calls to the Soviet Union frequently are broken. But Mrs. Ginzburg said she and her husband discussed his attempts in detail and have concluded it is part of the continuing harrassment of the family.

She said her visits to the embassy have been closely followed by plain-clothes police who on occasion photographed her returning home. The telephones of two friends of Knznetsov were removed by authorities after they received truncated calls from him, she said.

During one telephone call, Mrs. Ginzburg said her husband urged her to show their sons news photos of him so they would recognize him when they meet. The two boys, aged 6 and 4, have not seen their father since he was arrested in February 1977. Once thickly bearded with a full head of hair. Ginzburg was shaved and his hair was cropped short in prison and appears so in the most recent photos.

"They weren't sure they knew who he was," she said.

"I am ready to leave right now and want to go without any problem." she asserted. "but this is a terrible situation."