Two former officials of the Carter administration yesterday politely but firmly questioned the wisdom of the Reagan administration's newly declared policy of not negotiating with terrorists such as those who held American diplomats captive in Iran for more than a year.

Answering questions before the House Foreign Affairs Committee on the Iran hostage crisis, former secretary of state Edmund S. Muskie said it was "unwise to freeze in concrete" in advance the steps the country would take in response to a similar situation.

"The nonnegotiation option looks very easy not that the hostages are back" from Iran, Muskie said. "But if we had adopted that, the hostages would still be in Tehran."

Moreover, Muskie said, "those who say they would not negotiate have not suggested what they would do."

Muskie's position was supported by former deputy secretary of state Warren Christopher, the chief U.S. negotiator during the climatic stages of the crisis. He said suggestions that the United States should have set a deadline for the release of the hostages, to be followed by military action against Iran, would have backfired.

Christopher said such a course undoubtedly would have meant the deaths of the hostages since Iran, because of internal divisions and power struggles, was incapable of meeting any kind of U.S. deadline.

"It was a bad idea then and a bad idea now," he said.

House members' questioning of the two senior ex-officials was polite and the bipartisan praise for their individual performances during the crisis was lavish. But beneath the courtesy, there was an argument going on as Muskie and Christopher defended the Carter administration's handling of the hostage crisis in the fact of implicit criticism from their successors in the Reagan State Department.

That criticism was contained in a State Department announcement Wednesday that the new administration would honor and implement the terms of the hostage agreement that led to the freeing of the 52 American captives on the day President Reagan took office. "The present administration would not have negotiated with Iran for the release of the hostages," the State Department announcement said. "Future acts of state-sponsored terrorism against the United States will meet swift and sure punishment."

Reagan administration officials have refused to specify what they mean by "swift and sure punishment," but the threat of military action is implicit. Muskie yesterday questioned the wisdom of setting a deadline for taking military action, which he said would engage the United States in "the old familiar game of chicken" from which there could be no turning back.

"I don't think we have been diminished" by the willingness to end the hostage crisis through negotiations, Muskie said. "I think we might have been diminished if we had bombed [Iran]. It might have been seen as a misuse of power by one of the world's superpowers."

Muskie also said that U.S. allies "applaud the fact that, notwithstanding our raw power, we used restraint and were successful."

As they did earlier this week before the Senate Foreign Relations Committee, Muskie and Christopher defended the terms of the hostages agreement and were not seriously challenged on their assessment by committee members.

In response to questions about what preceded the downfall of the shah of Iran, Christopher said he thought U.S. intelligence services had underestimated the extent of the shah's political problems during 1978 and the deterioration of his own health during this same period.