An Albany, N.Y., theater company sang and danced onto the Soviet stage before an ebullient audience today, ending a U.S. boycott of the Soviet theater that began in 1979 when President Carter broke off the U.S.-Soviet cultural agreement in response to the Afghanistan invasion.

"Rag Dolly," an American musical remindful of "The Wizard of Oz," played to a packed house of more than a thousand, mostly children, at the Moscow Music Theater. "The singing," said 8-year-old Tanya, a Muscovite, "was just wonderful."

The Soviet news agency Tass concurred, billing the show "a great success" in a dispatch today. "The travel of American actors across the ocean is a mission of peace and love," Tass said.

Bursts of applause punctuated the matinee guest performance, and during the intermission a group of Soviet preteens broke into renditions of "We Shall Overcome" and "This Land Is My Land" in English. During the ongoing Soviet Christmas holidays, most of Moscow's theaters are offering special performances for children.

But the play was not widely advertised and tickets to the general public for today's guest performance were severely restricted.

The musical, written by William Gibson, author of "The Miracle Worker" and "Two for the Seesaw," is the first production under the new Soviet-American cultural exchange, signed at the November summit meeting between U.S. President Reagan and Soviet leader Mikhail Gorbachev in Geneva.

"If you were going to start a theater exchange between the two countries," said one Moscow director, "you could not have chosen a safer way to do it."

The production climaxes a two-year effort by the play's American director Patricia Snyder and the Soviet director of the Moscow Musical Theater, Natalia Sats, to bring the play to the Soviet capital. Sats made two scouting trips to Albany in 1984, including one in December with a Soviet Cultural Ministry official to see the play. "The whole thing would not have been possible without the cultural exchange," Snyder said.

The Moscow Music Theater plans to return the exchange with performances of "Peter and the Wolf" next June in Albany.

The eight performances here will mark the third run of "Rag Dolly" for the Empire State Institute for the Performing Arts, a professional theater company.

The play, a story about a young girl suffering from an illness who goes off in search of the doll doctor with a motley entourage of dolls, was performed almost completely in English but adapted for the Moscow stage.

Raggedy Ann, one of the lead characters, occasionally turns to the audience for quick translations of words and phrases into Russian, each time bringing beams and ovations from the crowd. At one point she sang the title song, "Rag Dolly," in Russian, inspiring a rolling thunder of applause.

But the play is chock-full of American colloquial expressions, most of which seemed over the heads of audience of preteens and their parents. At the start of the play, in response to a query by Sats, about 15 percent of the audience indicated a knowledge of English. "I know three words," said one girl in Russian, and flung up her hand.

Gag lines like "drop dead," or jokes playing on the difference between a Rolls-Royce and a Buick, or a running bowline and a granny knot seemed to hang in the air.

Inevitably, the crowd laughed at totally unexpected moments, said Ron Nicoll, a spokesman for the performance. And it broke into spontaneous clap-alongs when it was least expected, he added.

But on the whole the elaborate costumes and sets and the catchy musical tunes swept the audience and the troupe along.

Several young Russians left the play singing the Russian version of "Raggedy Ann."

"It was the first time I ever saw Americans on stage," Tanya said. "And I liked them."