Iranian militants holding American Embassy personnel hostage today rejected a mediation feeler by the Palestine Liberation Organization as U.S. hopes dimmed for negotiations on ending the five-day-old embassy occupation.

The militants adamantly continued to refuse to negotiate the release of the more than 60 American hostages unless Shah Mohammand Reza Pahlavi is returned to stand trial here from the United States, where he is undergoing cancer treatment in a New York hospital. The militants' position compromised any go-between role for the PLO and created visible tenisons in normally sunny Palestinian-Iranian relations.

[In Washington, State Department officials could not immediately confirm or deny a statement broadcast on the official Iranian radio that Bruce Laingen, U.S. charge d'affaires in Iran, had been arrested and placed under heavy guard. The radio did not say where Laingen, who was not in the embassy during the takeover, was seized or where he was taken. Since the occupation, he has been staying at the Iranian Foreign Ministry in Tehran.]

A Palestinian press agency communique announced the arrival of a delegation led by Saul Sayel, alias Abu Walid, the chief of the PLO's military operations center and a top aide of Yasser Arafat. But evidently embarrassed Palestinian officials pointedly demied PLO suggestions floated two days ago in New York that a mediation role was sought in what now was called "the Iranian revolution's internal affairs."

Further fueling Iranian militancy was the students' release of a purported State Department document alleging that the Carter administration was planning this summer to allow the shah into the United States before his deteriorating health was invoked to allow him to enter a New York hospital last month.

Hours after a blindfolded American male, apparently in his forties, was paraded on the embassy grounds near surging crowds chanting ant-American slogans, a student spokesman said, "We accept no negotiation whether by the PLO, [its chairman] Yasser Arafat, or by anyone else. Even if our brother Arafat suggested it, we will reject it.

"Our line," the spokesman told a news conference held at the embassey motor pool, "is the line of the Iman." He was referring to Ayatollah Khomeini, the country's spiritual leader.

Questions at the news conference were answered by several student representatives who spoke against a backdrop of four large color posters of Khomeini.

The students repeated the threat in apparent defiance of Khomeini's injunction, that they would kill the American hostages if the United States attempted to rescue them forcibly.

A Palestinian spokesman later showed irritation at the students' rebuff of Arafat's initial mediation offer.

"How can you say Arafat should keep his ideas to himself?" the PLO official said to reporters. "He is not a child; he's a chairman."

The new setback for negotiations followed Khomeini's refusal yesterday to receive an American delegation headed by former attorney general Ramsey Clark.

He had flown as far as neighboring Turkey before Khomeini suddenly decided to cancel a meeting scheduled by the country's ruling Revolutionary council.ionary Council.

Iranian insiders blame the cancellation on the Carter administration's decision to publicize the Clark mission.

The purported "secret -- eyes only" cable sent to charge d'affaires Laingen by Harry Precht, director of the State Department's office of Iranian affairs, was said to have been discovered after the embassy was occupied. Students complained that most of the classified documents had been shredded or otherwise destroyed.

In discussing possible approaches to justify allowing the shah into the country, the purported document, dated Aug. 2 argued that "we have resisted intense pressures to allow him to come to the "United States" in deference to "efforts to construct a new relationship" with Iran's revolutionary leadership. The nature of these pressures was not expanded upon.

Admitting the shah to the United States, according to the purported document, was to be timed for the end of 1979 when it was judged that the revolutionary leadership would have established "a stronger and legitimate authority for the country."

The cable noted that although threats to the embassy personnel had lessened since the spring, "the danger of hostages being taken will persist, especially if the shah were allowed to enter the United States.

"We should make no move towards admitting the shah until we have obtained and tested a new and substantially more effective guard force for the embassy."

"Secondly, when the decision is made to admit the shah, we should quietly assign additional American security guards to the embassy to provide protection for key personnel until the danger period is considered over."

Whether these precautions were taken in time was not immediately clear, but apparently the shah's sudden arrival in New York -- and the resulting uproar his presence there caused in Iran -- compromised the plan.

In Washington, State Department spokesman Hodding Carter refused to comment on the authenticity of the documents. He said, "I am not going to deal in any way with what promises to be a succession of statements coming from the people inside that embassy about alleged materials allegedly coming from the U.S. sources $.")

Reliable sources said privately the documents appeared to be genuine. But, they noted, the documents involved speculative recommendations tied to future events, and they pointed out that neither Laingen nor Precht has the authority to translate these ideas into actual policy $.)

The male hostage paraded around the embassy compound this morning was taken to within 20 yards of surging crowds restrained by the locked embassy gate.

His arms tied behind his back, his face covered with several days' growth of beard, the balding hostage was held for about five minutes in front of the crowd chanting in English, "Death to Carter" and "yankee Go Home." One student said the man had been singled out for punishment because he "refused to give us his name. We brought him to listen to the shouts so that he'll realize we are not alone."

However, during the day, other student spokesmen said the hostages, who are believed kept in various parts of the sprawling embassy compound in central Tehran, were "quite healthy."

At the request of the International Red Cross, the hostages had been visited by an Iranian doctor acting under the auspices of the local equivalent of the Red Cross known as the Red Lion and Sun Society.

However, student spokesmen dodged reporters' questions when asked why they had failed to honor an earlier pledge to allow the press to visit the hostages.

Photographs handed out at the news conference showed several hostages lounging in the embassy. Earlier, students explained that "hostages were blindfold only when we don't want them to see something,"

He added, "They were laughing and joking until yesterday when Ayatollah Sadegh Khalkahali came to visit us. They haven't laughed so much since."

Khalkhali nicknamed by critics "judge Blood," is a notorious roving official of the revolutionary tribunal who claims to have personally ordered the execution of at least 30 Iranians since the revolution in February.

Meanwhile, informed sources said the Revolutionary Council -- which replaced ousted Prime Minister Mehdi Bazargan's government Tuesday -- has been expanded from nine to 14 members. The expansion was ordered now that the council is to assume governmental as well as "guidance" responsibility. Eight of the members are laymen, and six are religious figures.

Picked as secretary general -- the top coordinating job now that the prime minister's position has been abolished -- is Ayatollah Mohammed Behesti. His special responsibilities will be housing and generally improving the lot of the poorest Iranians.

Replacing outgoing foreign minister Ibraham Yazdi is said to be Abol Hassan Bani-Sadr -- a long-time resident of France and economic planner who is to oversee eight economic portfolios.

Ayatollah Hashem Rafsanjani and close Khomeini associate Mahdavi Kani, chief of the influential Islamic revolutionary committees, are likely to share the interior, security and administrative responsibilities.

A clergyman identified as Ayatollah Khameni, little known in the republic but a force inside the Revolutionary Council, is slated to take over the defense portfolio.

In other developments, the students announced that a special squad had been sent to the Foreign Ministry where Laingen and two aides have been staying since the embassy was occupied Sunday. A student communique said Laingen was "considered a spy" on the basis of the documents found in the embassy and "is considered a hostage and does not have the right to leave the country."

The national Pars News Agency reported that a young Iranian who last night tried to immolate himself outside the embassy was in critical condition. The agency said he had left a letter explaining his act was to protest the shah's continued stay in the United States.