Despite the widespread public impression that most Iranian diplomats here have been ordered to leave the country, the State Department said yesterday that few, if any, have actually left and it is uncertain whether any will depart in the near future.

The order announced Dec. 12 called on Iran to cut down its representation in this country from 218 accredited diplomatic personnel to only 35, with a deadline of last Monday, Dec. 17. The action was taken "to demonstrate to the government of Iran our continuing concern over the illegal holding of hostages and American property in Iran," it was announced last week.

State Department officials said the Iranian Embassy furnished a list Wednesday of 34 officials who have been designated to remain. At the same time, spokesman Hodding Carter said it is "murky" whether and when the vast majority of the diplomatic personnel will leave.

A substantial number of the Iranians are married to Americans or are long-term residents pressed into service temporarily by the Iranian regime after the downfall of the shah, according to the State Department. These people have a claim to remain here on other grounds, even though they lose their diplomatic status.

Other Iranian diplomatic personnel may wish not to return, on grounds that they may face persecution at home, according to the State Department. However, there is no report that any current Iranian diplomats have sought political asylum.

Those who intend to leave have been granted "a decent interval" to get their affairs in order, Carter said.

He said the slow-moving and murky situation is due to "our courts, our own laws, the procedures of a civilized country and the difficulties of sorting out who is who" in a democratic society.

In his daily briefing for reporters, Carter also said that the possibility of a Soviet veto or an unfavorable vote from other nations in the U.n. Security Council would not deter the United States from seeking economic sanctions against Iran.

No such decision has been made, he said, but it is among the options under consideration. Some officials believe President Carter will authorize a drive for United Nations sanctions in the next few days.

"The United States intends to seek its goals through every appropriate legal avenue . . . whatever we think of the chances of success," the State Department spokesman said.

Soviet Ambassador Anatoliy F. Dobrynin had been expected to return from Moscow this week with an answer toU.S. feelers about economic sanctions on Iran. But Soviet Embassy officials said Dobrynin is not back, and they have no indications when he will return.

The State Department, reversing last week's strong criticism, praised Japan for cooperating with measures to put pressure on Iran. Spokesman Hodding Carter noted recent statements by Japan's Prime Minister Masayashi Ohira, as well as new actions to prohibit in influx of Iranian oil in Japan. Carter said the United States has been assured that "Japan will keep step with European countries" in applying economic pressure.

While the administration continues to expound its policy, a former high official of the Nixon, Ford and Carter administrations was sharply critical of the U.S. failure to take stronger action in the early days of the crisis. o

James R. Schlesinger, who served as CIA director, defense secretary and energy secretary, among other posts, told reporters that the United States should have set a deadline for release of the hostages and threatened Iran with "punishment" if the deadline was not met.

Asked about the risks to the hostages and the U.S. stakes in the Middle East, Schlesinger declared that "the greatest risks are those associated with inaction." He said that a strong U.S. military force should have been dispatched to the region at the same time the initial ultimatum was delivered.