The U.N. Security Council called on Iran today to release immediately its 50 American hostages and to use the United Nations as a forum for resolving its dispute with the United States. u

In a unanimously approved resolution, which many diplomats see as the last opportunity for peaceful resolution of the crisis, the council appealed to Iran and the United States to settle their differences by peaceful means and authorized Secretary General Kurt Waldheim to provide his "good offices" toward that end.

The resolution was approved by the 15-nation-council after 4 days of emergency debate and backstage discussions. It condemns the seizure of the U.S. embassy in Tehran yet opens the way for Iran's acceptance of U.N. mediation.

The wording of the document aims at providing the Iranian authorities with an opportunity to get off their current collision course with the United States without losing face.

Its effect was to make Waldhiem, for the moment at least, the principal figure in the thus far unsuccessful attempt to open an effective channel of communication and negotiation between the Carter administration and Ayatollah Ruhollah Khomeini's revolutionary government in Iran.

At present, there is no way of prediciting whether Waldheim will have any more success than the many other governments, organizations and individuals that have tried unsuccessfully to act as intermediaries.

[A spokesman for Moslem militants occupying the U.S. Embassy dismissed the resolution, saying "we do not recognize the Security Council," and adding: "Peaceful measures mean the extradition of the shah," Reuter reported from Tehran. A spokesman for the Iranian Foreign Ministry, Ibrahim Mokalla, said only that the U.N. action would have to be discussed by the Revolutionary Council, which in theory runs Iran].

Two previous appeals by the council for release of the hostages, expressed in the form of statements by the council chairman, have been ignored by Iran.

The Iranians also boycotted the council's emergency meeting on the crisis, and Khomeini has said he will ignore any actions or appeals by the council. During the debate here, the Iranians gave no sign of backing away from their demand that the United States forcibly return deposed Shah Mohammad Reza Pahliavi to Iran to stand trial.

Waldheim underscored the difficulties he faces when, in a brief statement at the end of the meeting, he noted that "there are few if any precedents" for dealing with the situation and said; "Innovation may be necessary."

Unlike two earlier appeals, today's measure was a formal resolution endorsed by all 15 members in a recorded vote. Its adoption was preceded by heavily publicized debate that saw countries representing every region of the world and every shade of political and ideological belief agree unanimously that Iran has violated diplomatic law by using diplomats as hostages.

That marked a major, tactical step forward for the Carter administration's strategy of trying to secure the captives' freedom through restraint and patient diplomacy rather than resorting to threats and precipitate action.

In addition, the meeting enabled the United States to block Iranian attempts to shift the focus of world attention away from the plight of the hostages to Iran's grievances against Washington and the shah.

As U.S. Ambassador Donald McHenry said at the end of the meeting, "It is clear from the vote and the debate that the family of nations speaks with one voice in calling for the release of the hostages."

Besides to putting Iran on notice that its actions have isolated it diplomatically, the council meeting, coupled with Washington's pending appeal to the International Court of Justice in The Hague, have also put on the record U.S. willingness to use every available forum to find a peaceful way out of the dispute.

Building such a record could prove of major importance later if diplomacy fails and the United States has to ask the United Nations for tougher action, such as imposition of sanctions against Iran. In a worst-case scenario involving the possible use of military action, it also could be cited as proof by the United States that it had exhausted all other available remedies.

Although President Carter has stressed continually that he is not contemplating military measures, the possibility that the United States might eventually have to turn to a tougher stance created what diplomatic sources here say was the major stumbling block in the back negotiations on the resolution.

Several Third World members of the council wanted the resolution to state a specific prohibition against the use of force or interference in the internal affairs or territorial integrity of any nation.

The prohibition was opposed by the United States on the grounds that it could block a possible later American resort to Article 51 of the U.N. Charter, which gives member nations the right of self-defense against attack.

The matter was compromised by putting language into the preamble of the resolution that reminded member states of their "responsibility" to act with restraint in disputes but that also referred to acting "in any other manner inconsistent with the purposes of the United Nations." That is regarded by U.S. officials as clearing the way for invoking Article 51 if the United States should deem it necessary.

Most diplomats here feel that the best hope for an effective U.N. role lies in reviving Waldheim's dormant efforts to work out a three-part deal -- release of the hostages in exchange for Iran receiving the opportunity to state its grievances before the world body and appointment of a commission to investigate the shah's alleged crimes.