President Reagan chose the lowest option in the retaliation list his aides gave him against Iran's Silkworm missile attack last Friday in hopes of slowing the rapid pace up the escalation ladder, but that goal may prove to be as slippery as getting Iran to stop its war against Iraq.

Reagan's bloodless response cost no lives. It also served notice on Tehran that the United States was willing to limit its little war with Iran to maritime regions. The clear message: the United States can show restraint if you do. But the radical regime in Iran, with the ailing Ayatollah Khomeini more unstable than ever, may not reciprocate.

If the Iranians aim their next Silkworm at Kuwait's $5 billion Al Ahmadi port complex with its oil refinery and gas condensation plants, the explosion could kill tens of thousands of Kuwaitis. That would terminate modulated reprisals and open a new phase of the Gulf war: disengagement or massive retaliation.

The question facing Reagan and his advisers last weekend was not whether to retaliate, but how strongly. Most of the options National Security Adviser Frank Carlucci gave the president differed in degree only, with no top official opposed to some form of military retaliation.

Rather, the problem deals with what the president was hoping to accomplish when he decided not to ''sink a ship'': finding some way to stop or slow escalation in the confrontation between superpower America and fanatical Iran.

If Reagan and national security hard-liners see danger in the uncontrolled escalation Iran seems determined to thrust on the United States, liberals are livid. One key Democrat on Capitol Hill, always against reflagging Kuwaiti tankers to give them protection in international waters, told us privately that ''escalation is now out of our hands and completely within the whim of a frenzied Iran.''

The administration's conviction last July when the reflagging operation was getting under way was quite different. The intelligence community saw it differently, advising Reagan there would probably be no quick Iranian retaliation for reflagging or against the Navy buildup. Moreover, if and when the ayatollah struck back, Reagan was advised, it would be ''tangential,'' with actions that did not leave clear fingerprints -- such as hostage-taking and terror.

When Iranian buzz-boats fired at a U.S. helicopter two weeks ago, fingerprints were everywhere. Then last week, with Iranian intelligence in Kuwait operating at high efficiency, came the Silkworm attack on the Sea Isle City. It could not have been an accident.

That shows the difficulty of Reagan's continuing his policy of modulated retaliation, even if one of its explanations is Khomeini's latest illness -- a potentially fatal brain tumor that has him now surrounded with Austrian doctors imported to save his life. If his much-predicted demise is actually at hand, low-level retaliation might ease U.S. relations with a successor regime, while reprisals costing Iranian lives could do the opposite.

Two weekend New York Times dispatches claiming the chairman of the Joint Chiefs of Staff, Adm. William Crowe, opposed hitting the Silkworm site on the Faw peninsula were contradicted by Capt. Jay Coup, his special assistant. Coup told us Crowe's position was based on the fact the launch site was abandoned within hours after the missile had been fired and could not have been targeted. Led by Crowe, the JCS proposed that the president ''sink a ship,'' one of the toughest options handed him after the U.S.-flagged tanker was struck in Kuwaiti waters.

With forewarning to the Iranian specialists who operated the high-tech radar equipment on the Rostam platform demolished by U.S. naval gunfire, no lives were lost. But the president's top advisers know that such tender, bloodless reprisals are strictly for the short range. No one is betting Tehran will leash itself.