Iran yesterday relayed a new set of questions about the latest U.S. proposals on release of the 52 hostages to Washington through Algerian mediators, White House Press Secretary Jody Powell said last night.

State Department officials immediately went into an unusual late-night meeting to discuss the questions, in a move that touched off intense speculation here that Iran was prepared to take another step in the negotiations that began in early November.

President Carter interrupted a White House dinner for Democratic governors and mayors to meet with Deputy Secretary of State Warren Christopher in the Oval Office on the developments. A reply was sent to Iran through the Algerian intermediaries shortly before midnight.

In Tehran yesterday, Iranian Foreign Minister Mohammed Ali Rajai told Iranian television that Ayatollah Ruhollah Khomeini had authorized the government to accept unspecified guarantees by Algeria "to solve our problem with the United States," according to news agencies.

There was no elaboration of Rajai's remarks after his meeting with Khomeini and no explanation of what he meant by Algerian guarantees.

Officials in Washington said they did not know the exact meaning of Rajai's comments and they refused to disclose the substance of the questions they had received. But there were indications that the questions dealt in part with the amount of frozen Iranian funds the United States was prepared to turn over at the time of the hostage release.

The latest American proposal, according to administration officials, includes return to Iran of $4.5 billion of their more than $8 billion in frozen assets plus another 1.6 million ounces of their gold.

In addition, when the hostages are released, the United States is prepared to return the rest of the frozen assets when the Iranians agree to participate in a settlement commission that will include binding arbitration of U.S. corporate and individual claims against the Tehran government.

Tehran radio said yesterday that the head of the Iranian hostage negotiating commission, Behzad Nabavi, would hold a news conference today -- normally considered a sign that some new Iranian position is about to be annonced to the Iranian people.

One White House official said yesterday's diplomatic report from Tehran was another step toward bringing the negotiations to a conclusion. But, he added, "We still have, among other things, to reach some final agreement on numbers and implementation."

In addition, the U.S. officials involved in the negotiations expect that they will have to take to the Supreme Court any agreement that requires transfer of American claims against Iran tribunal. That process alone, sources said, would take at least two weeks.

Some Washington officials cautioned late yesterday evening that the early reports from Tehran may not hold up.

News agencies also reported the following from Tehran yesterday:

According to one report on Iranian television, Rajai said: "We asked the imam [Khomeini] about the hostages and we explained the new opinion of the U.S. government to the imam and also we explained the Algerian proposal which has suggested that it will guarantee to solve our problem with the United States, and the imam permitted us to accept these guarantees and we hope to annouce the rest of the points."

Khomeini, in a long statement on the meeting with the Cabinent yesterday, made no mention of the hostage issue.

Secretary of State Edmund S. Muskie was quoted by Associated Press last night as saying that Khomeini's statement "sounds positive. On the other hand, there have been other references in Tehran to statements by Iranians to the effect that guarantees approved by the Algerians would be sufficient. But we don't know what those statements mean."

The word of new questions from Tehran and the reports of Khomeini's endorsement of the Algerian role touched off a spate of rumors and broadcast reports of an imminent breakthrough toward release of the Americans.

A team of U.S. officials headed by Christopher met at the State Department in an unusual evening session to formulate the answers to the questions raised by Iran.

This meeting coincided with news reports, denied by members of the U.S. policey-making team, that the Algerians had informed Washington that Tehran "accepted in principle" the most recent U.S. proposals.

Reporters and television teams descended on the State Department. Despite a paucity of reliable information, the speedup in diplomatic activity and the welter of rumors created an atmosphere of drama in Washington.

Powell, one of several officials who cautioned against expecting a decisive development, said, "I don't think there's any basis to be optimistic, and certainly none to be more pressimistic than we have been. We just have to wait and see."

The effect of the statemnts in Tehran and Washington yesterday was to underscore that the Algerians are now playing the central role in the negotiations.

Although the Iranians have described them as assuming a role as guarantors of any agreement, the Algerians, according to Washington sources, have so far avoided assuming any role beyond that of intermediaries.

In Tehran yesterday, Algerian Ambassador Abedel Karim Gheraieb denied that there had been any change in his country's role as intermediary, Agence France-Presse reported.

Despite what Gheraieb said in Tehran, the prospect that the Algerians would assume a role as guarantors of any agreement, instead of just intermediaries, first was established publicly last month by Iran.

In two key portions of the Iranian reponse sent to Washington Dec. 19, Algeria was specifically described as being able to provide guarantees if the United States failed to provide the funds sought by Iran.

For example, on the most controversial cash demand of the Iranians -- the $10 billion to cover alleged property of the shah and his family in the United States -- the Tehran government note said that it would accept "any other guarantee acceptable to the Algerian government."

At another point in that proposal, the Iranians wanted Washington to deposit a guarantee equal to $4 billion in cash to cover its frozen assets other than cash being held by American banks and corporations.

The Iranian note added, however, that if the $4 billion were not deposited, it could be replaced with "any other valid guarantee acceptable to the [Central Bank of Algeria]."

A Washington officials familiar with the negotiations believes the Algerian role was approved by Khomeini as long ago as early November when the first Iranian position was delivered.

"Otherwise the Algerians would not have been allowed into the process in the first place," he said.

American negotiators, too, are looking to the Algerians as guarantors -- in this case to make certain the Iranians will live up to any third-party claims procedure that is established and pay any awards to American companies or individuals made by mutual agreement or arbitration.