The Soviet Union's showcase International Book Fair opening Tuesday has already been marred by official Soviet censorship and U.S. cancelation of a gala reception for the Soviets as a protest over Moscow's refusal to issue a visa for a U.S. publisher.

Soviet customs men today banned U.S. publishers including five works by exiled novelist Alexander Solzhenitsyn, two by Josf Stalin's daughter Svetlana Alleluyeva who fled the Soviet Union 12 years ago and a collection of caricatures by New York political cartoonist David Levine.

Meanwhile, the U.S. Embassy disclosed tonight it has taken what an official called the "very unusual step" of cancelling a reception scheduled for next Friday. The reception was to be cohosted by the Association of American Publishers and Ambassador Malcolm E. Toon at the envoy's glittering official residence Spaso House.

The cancelation was sought originally by the AAP to protest the Soviet refusal to issue a visa to Robert Bernstein president of Random House Books and an outspoken critic of human-rights violations by the Kremlin.

The visa turndown already has been protested to no avail by the State Department. The heads of Ardis Press in Ann Arbor, Mich., a U.S. publishing house specializing in Russian literature, also have been refused visas. Carl Proffer and his wife, who run Ardis, recently published in facsimile edition an unofficial collection of prose and poetry, "Vetropol," written by Soviet authors seeking to challenge Soviet censorship.

Hundreds of Publishers from all over the world will present their work at the fair, the second such international Moscow book exhibition since 1977. The official Soviet media is hailing the fair as a fresh demonstration of Moscow's position as a mecca for all those interested in freedom of expression and the free exchange of ideas. But the banning of works considered ideologically dangerous or by authors whose words have been suppressed has already exceeded the eight U.S. titles seized at the 1977 fair.

U.S. exhibit officials predicted tonight the number of banned books will grow. Customs officials plowing through the hundreds of cartons of newly arrived books at Pavilion No. 2 of the International Exposition Grounds where the fair is being held have taken another nine works for "review by higher authority."

These include five different volumes of "Best Editorial Cartoons of the Year," as well as books about Judaism, Nazi Germany, U.S. foreign policy and U.S. impressions of the Soviet Union.

The works banned outright include "Bravo Baryshnikov," a biography of the Kirov Ballet star who defected to the West and "Bukharin," by Stephen E. Cohen, a biography of one of the founders of the Soviet state whose reformist economic and political ideas came to be banned here.

Also banned were Israeli Prime Minister Menachem Begin's memoirs on his time in Stalin's prison camps, "White Nights;" "The Arts of David Levine;" Solzhenitsyn's three-volume "Gulag Archipelago," "The First Circle," and his 1978 commencement address at Harvard entitled "World Split Apart," and Alleluyeva's "Twenty Letters to a Friend" and "Only One Year."

The fair is sure to attract tens of thousands of Russinas eager to sample the wares of the world's publishers. In 1977, the crowds were heaviest at the U.S. and Israeli booths, as well as at specialist art publishers whose high-quality color reproductions cannot be obtained here.

The centerpiece of the U.S. effort will be a 321-title AAP exhibit called "America Through American Eyes," consisting of recently published books chosen by a panel headed by novelist Kurt Vonnegut Jr. to reflect contemporary U.S. life. The panel's selections are to be announced Tuesday by AAP officials, but Soviet censors have already winnowed through these samples of U.S. thought and found something to object to: Levine's banned book of cartoons was to be part of the special exhibit.