For the second time this year, mob action in Iran has shattered the Carter admistration's perceptions of its ability to deal with the Iranian revolution by staying close to the government in power.

Only a month ago, middle-level State Department officials were encouraging U.S. businessmen to go back to Iran and work with the Islamic revolutionary regime of Ayatollah Ruhollah Khomeini in a bid to renew the American presence.

Today, some of those same officials are working desperately to stave off a human disaster involving Americans in Iran. As many as 65 Americans are facing a death threat in an embassy thatWashington has been building up in size in recent months. Hundreds of U.S. businessmen are on the verge of fleeing again or have gone into hiding.

The official encouragement to Americans to return to Iran was part of a policy of accomodation with the new Iranian leadership in an effort to overcome the stigma of Washington's much warmer relationship with the deposed shah, whom Carter continued to support until continuing protests drove him from power in January.

Until the seizure of the U.S. embassy for the second time since the revolution -- and this time with Khomeini's expressed blessing -- American policy toward the Islamic Republic was based on the administration's stated perception that the Iranian government wanted to improve relations with the United States and that anti-Americanism in Iran had been exaggerated.

Besides creating another crisis for the Carter administration, the mob takeover of the U.S. embassy in Tehran Sunday may finally shatter the illusion on which American relations with both the shah and the Khomeini regime have been based.

That illusion is that by becoming, to an extent, apologists for whoever holds power in Iran and by promoting the export there of American citizens, technology and culture, Washington can wield influence with a basically xenophonic nation and preserve U.S. strategic interests in a volatile part of the world. Those interests can be summarized as keeping the oil flowing and the Russians at bay.

By the time the shah's monarchy began to crumble last year, massive U.S. arms sales, close cooperation between the two countries's armed forces and intelligence agencies, the presence of more than 40,000 Americans in the country and repeated statements of support for the shah's policies had inextricably bound the U.S. government to the outgoing regime.

Besides making the United States public enemy number one of the new regime, the years of U.S. support for a dictoral system helped create the current circumstances in which democratic Iranian forces are weak and disorganized, enabling the new clerical rulers to step into the shah's place.

Nevertheless, after the revolution a U.S. administration that had turned a blind eye to human rights violations and other excesses under the shah began to bend over backwards in dealing with the new regime to avoid offending the ruling clergy. The administration was anxious to reach a new accomodation, seemingly ignoring the circumstances that fostered the revolution and the likelihood that growing discontent with clerical rule would eventually bring down the new regime and risk another setback for U.S. policy.

As recently as September, administration officials dealing with Iran stressed this desire for closer relations in a background briefing for reporters. Reflecting Iranian criticism of Western reporting, the journalists were told that "irritations" in the U.S.-Iranian relationship "need to be balanced by an appreciation of our long-range interests."

One official said the State Department was "encouraging American businessmen to go back" to Iran because this was "important for our national interests."

Stressing that the United States accepted the Iranian revolution, the officials argued that the Islamic Republic was solidly in place and that "Islam . . . is not necessarily a bad force."

The administration's message boiled down to this: that Americans should not let the "irritations" -- the executions, Khomeini's rartings against the United States the expulsions of journalists, the increasingly dictatorial rule of the clergy -- cloud their perceptions of "sharp interests" between the two countries.

As a result of this policy, the administration contributed significantly to the current U.S. vulnerability in Iran.

In its desire to improve relations, despite clear statements by Khomeini of continuing hatred and suspicion of the United States, the State Department built up the embassy in Tehran from a low of about 30 staffers immediately after the revolution to about 75 at the time of the latest embassy takeover.

Until last month the State Department also was urging the nomination of a new ambassador to Iran whose first duty, officials said, would be to make the pilgrimage to Khomeini's residence in the holy city of Qom to try to open up a line of communication to the ayatollah. Already, the Soviet ambassador had come in for a dressing down in at least one such audience, and U.S. embassy officials were predicting worse for any American envoy.

At the same time that the administration was trying to accommodate Khomeini, however, some junior diplomats in the Tehran embassy were privately cautioning against a higher U.S. profile.

The only reason the Iranian government had not already cut diplomatic relations with Washington, one diplomat said, was because Iranian authorities wanted the embassy to keep issueing visas to thousands of students and other Iranians, including members of the new government, who want to visit or study in the United States.