IT IS TRULY unfortunate that, by his previous deeds, Jimmy Carter had not established in the minds of the Iranian authorities that he and his government could not be trifled with and that the Iranians could not evade their elementary responsibility to treat American diplomats by the time-honored rules. It is no less regrettable that the administration had not anticipated more carefully the dangers posed to its diplomats by the disintegration of the local government's authority and the ascendancy of the Ayatollah Khomeini's mob. The painful result is that Mr. Carter seems to have been reduced to reliance on the PLO, which the United States is pledged not to "recognize or negotiate with" and which is in a position to demand that he pay dearly for the access it enjoys to Imam Khomeini. In brief, the PLO offers him the prospect of escape from one dilemma in return for getting caught in another.

The PLO cannot be faulted for capitalizing on a windfall, which in any case comes at a moment of gathering American interest in bringing the PLO into Middle East negotiations. It sees an opportunity to break through to American opinion and the American government without having to suffer the political inconvenience of moving toward the Israelis at the same time. So, it is that the PLO has been teasing the administration, explaining that it needs to be compensated for the risks it incurs in approaching the ayatollah and suggesting that it would like to be directly asked to intercede, preferably by the president himself. The administration appears to be looking for the formula that will let it keep its no-negotiations pledge to Israel -- a pledge made by President Ford and reaffirmed by President Carter -- without letting the PLO slip off the hook.

Actually, we are sure that the Israelis will accept an obviously humanitarian exception to the no-negotiations rule. But two caveats must be registered. The PLO is, in one of the aspects, a terrorist organization. The other day an Israeli court convicted two Palestinians who, in search of hostages, had precipitated a massacre of 34 Israelis. For the PLO to receive uncritical credit for trying to negotiate the release of American hostages, while it still pursues a policy of taking -- and murdering -- Israeli hostages, would be outrageous.

Then, the real political rewards that the PLO ought to be able to expect from the United State should be for shifts in its policy toward Israel. If the PLO takes the Tehran case, everyone will be rooting for its success in the mission. But the administration, having been humiliated by one party in the Mideast, cannot possibly invite humiliation by another.