U.S. officials in Moscow have held discussions with Svetlana Alliluyeva, the only daughter of Joseph Stalin, but the State Department yesterday declined to comment on a report that she is seeking to return to the West.

The department confirmed it had been talking to Alliluyeva, who returned to the Soviet Union in late 1984 after living in the West for 17 years, after the Sunday Times of London reported today that Alliluyeva wants to come to the United States to join her daughter, Olga.

The newspaper said Olga recently moved back to the United States following a secret deal with Soviet authorities.

But Olga's father, architect William Wesley Peters, said in a telephone interview last night that he "questioned whether the report was true."

"I really know nothing about it," he said. "I think if it was true, that Olga was here, I would know about it."

The State Department declined to discuss the newspaper report. Spokeswoman Anita Stockman said, "Our embassy in Moscow has been in direct contact with Svetlana Alliluyeva. In accordance with the provisions of the Privacy Act, we do not comment on consular discussions with American citizens."

Alliluyeva's defection in 1967 caused a worldwide sensation and was a propaganda blow to the Kremlin. Her return to the Soviet Union also created a stir, with Alliluyeva saying she had not enjoyed "one single day" in the West and the Presidium of the Supreme Soviet publicly restoring her Soviet citizenship.

Her return had important political aspects for the Kremlin, since it came during planning for the 40th anniversary of the Soviet victory over Nazi Germany, and at a time when Stalin was being rehabilitated as a great leader and diplomat.

Olga -- 13 years old when she went to the Soviet Union with her mother -- was born in the United States to Peters and Alliluyeva, who married in 1970 and were divorced two years later.

The Sunday Times of London, as quoted by Reuter, said that "since her daughter's departure for the United States, Svetlana's unhappiness has apparently deepened. This led her to apply for permission to leave Russia and go back to America."

Peters, reached at Taliesin West, a part of the Frank Lloyd Wright foundation north of Phoenix, said he was "very anxious to see Olga, and I'm hopeful that she'll come back here."

He declined to discuss his contacts with Alliluyeva and Olga, saying they were made "with difficulty, on a second- and third-hand basis."

After Alliluyeva, who uses her mother's maiden name, and Olga returned to the Soviet Union, they settled in her father's Georgian homeland, where she had relatives through a late stepbrother