The U.N. inquiry commission on Iran yesterday examined documents in Geneva alleging human rights violations under the deposed shah, but potentially ominous signals from Iran clouded the panel's plans to leave for Tehran this weekend.

Despite a hard-line statement from the militant captors holding American hostages in Tehran and renewed calls from Iranian authorities for extradition of the former shah, U.S. and U.N. officials believe the inquiry panel remains a workable method for settling the 110-day-old crisis.

Officials pointed out that the strident demands for the former shah's return made by Ayatollah Ruhollah Khomeini and President Abol Hassan Bani-Sadr fall short of past demands linking the monarch's extradition to the release of the American hostages in Tehran.

In Tehran, however, the militant captors of the U.S. Embassy continued to link any release to the shah's return, saying that the United States is "foolish" if it expects the hostages to receive "clemency" before the "fugitive" deposed shah is handed back to Iranian authorities.

A spokesman for the militants told Reuter that they were encouraged by an appeal from Khomeini Wednesday for the nation to Forcefully" press for the shah's return. Khomeini, Iran's religious leader, called upon God to support the Islamic militants.

But diplomatic sources largely discounted the Khomeini statement as a possible attempt to flatter the militants. As the only Iranian leader with control over the radicals, sources said, he may be preparing them for an order to free the Americans.

Any Khomeini call for the hostage release, however, would come after the five-member inquiry commission arrives in Tehran and begins its investigation of alleged crimes by the deposed shah. Mohammad Reza Pahlavi, according to sources familiar with the panel.

Commission members met yesterday with U.N. human rights officials and representatives of the International Committee of the Red Cross and the International Commission of Jurists, two organizations that have studied the Iranian human rights situation under the shah. They received documents on Iran in the afternoon.

The commission had expected to leave Geneva for Tehran Wednesday, but changed its plans at the last minute after Iranian authorities asked U.N. Secretary General Kurt Waldheim to delay the mission until this weekend.

U.N. officials said the delay was "purely technical," to allow Iran to prepare for the hearings that the commission is expected to hold to receive allegations against Pahlavi and the United States, which supported his government.

But knowledgeable sources believe that Bani-Sadr asked for more time to prepare public opinion of the eventual release of the Americans and to guarantee that Khomeini would follow through on his promise to direct the militants to free the hostages.

Some sources said the delay was caused by a remaining dispute about the commission's mandate. U.S. officials had sought to limit the panel to an inquiry into the shah's rule, but the mandate was left open ended to allow for a broader investigation.

Further speculation that the Waldheim mediation mission is in trouble arose when commission cochairmen Mohammed Bedjaoui of Algeria left Geneva for New York yesterday morning with plans to see Waldheim today to discuss the panel's work.

A U.N. spokesman insisted that Bedjaoui, who is his country's ambassador to the United Nations, returned on "official business" unrelated to the commission. But knowledgeable sources questioned whether he would make the trip unless the inquiry had run into snags.

Bedjoui said Wednesday that there was a "gentlemen's agreement" that the estimated 50 American hostages in the U.S. Embassy in Tehran and three U.S. officials in the Iranian Foreign Ministry would be freed as a result of the U.N. investigation.

But Bani-Sadr refused to discuss a possible release in an interview with a French Canadian radio station yesterday and repeated his government's determination to extradite the ex-shah, who now lives in Panama.

In past remarks, the Iranian president has indicated that the extradition of Pahlavi is a long-range goal of his nation and is not related to the release of American hostages, who have been in captivity since Nov. 4.

Diplomatic sources say that Waldheim has received general assurances from the Iranians that the hostages would be freed an some stage of the inquiry possibly after the commission completes its work. The first sign that plans were going ahead would come with a Khomeini endorsement.

In Tehran, Iranian Foreign Minister Sadegh Ghotbzadeh, who negotiated with Waldheim on the commission and its guidelines, said in an interview with the official Pars news agency that the panel would arrive "early next week."