The United States called on the U.N. Security Council today to issue a public warning to Iran that the trial of any American diplomatic hostages on spy charges would constitute a violation of international law.

In a separate move, Secretary General Kurt Waldheim, after consultation with Washington, issued a statement expressing his "deep concern" about the possibility that the hostages would be tried by Iranian authorities. a

"If such a trial would take place," Waldheim said, "it would be a violation of the Vienna conventions and diplomatic immunity."

These U.S. moves were designed to put the international community on record as opposed to the trial of any of the hostages in advance of any such action by the Iranian government.

Council members have already indicated in private their endorsement of the principle involved. But some have expressed reservations about the issuing of a statement on their behalf by this month's president of the Council, Sergio Palacios de Vizzio of Bolivia.

A similar statement, appealing for the release of the hostages, was issued on Nov. 9, but the Iranians have never replied. Instead, they have asked for a public debate in the Council to discuss the "war psychology" that they maintain is being promoted by the U.S. government.

Council members have not rejected this Iranian request, but have stonewalled by holding daily consultations.

[Iran has sent a special envoy to the U.N. to press for a Security Council meeting, United Press International reported. Iranian Foreign Minister Abol Hassan Bani-Sadr dispatched Hamad Salamatian to New York to insist that the council hold a debate on U.S. policy toward his country, the agency said.]

De Vizzio said he expected the Council to consider issuing another presidential statement at a closed meeting of all 15 members tomorrow morning.

Soviet Ambassador Oleg Troyanovsky told reporters that, in principle, the trial of diplomats would be, like the holding of diplomatic hostages, a violation of international law. "But the question is what would be the best way for the Council to act in this situation," he said.

U.N. officials said they believe the Council members would accede to the American request tomorrow, but, "it is not clear that such an appeal would have any impact," one official pointed out.

An American legal expert here said that all governments represented on the Security Council take the position that the trial threat posed to the hostages is a violation of the Vienna conventions on diplomtic privileges and immunities, to which Iran and virtually all nations subscribe.

"No one here would offer a reasonable, sustainable quibble about that," the American expert said. "One might get some grumbles about the wisdom of issuing such a statement," he went on, "but assuming that the language the Security Council president proposes isn't pejorative, and constitutes a further appeal to the Iranian authorites, I think our chances of winning Council action are good."

The only thing that can legally be done to them, if they are charged with espionage, is to expel them, he said. The legal expert cited the case of three Soviet nationals charged with espionage in a New Jersey court case this year. Two were tried and convicted because, as U.N. Secretariat members, they were not covered by the Vienna conventions. The third -- a member of the Soviet mission to the United Nations, was not put on trial, but was declared persona non grata and deported, because of his diplomatic immunity.