In the first of a new series of steps designed to demonstrate U.S. activism, in the Iran crisis, the State Department yesterday ordered all but 35 of the 218 Iranian diplomats in this country to leave within five days.

Spokesman Thomas Reston announced that the diplomats were being ordered out "in a view of the continued illegal detention of American personnel and the holding of the American Embassy compound at Tehran as well as U.S. government property at Tabriz and Shiraz" in Iran.

Reston called the action "a measured step to demonstrate to the government of Iran our continuing concern over the illegal holding of hostages and American property in Iran."

Official sources said that other steps and statements can be expected in the coming days to keep the consequences of their actions before the various groups in authority in Iran, and to maintain a sense of U.S. initiative in the view of the American public.

The actions ordered yesterday will reduce the Iranian Embassy here from 57 diplomatic employes to 15. The Iranian consulates in New York, San Francisco, Chicago, and Houston, which have 161 diplomats and diplomatic employes, are being required to reduce to five persons each.

It will be up to Iran to decide which diplomats must leave. The reductions will be in effect as long as the crisis lasts, the State Department said.

Consideration has been given to closing the Iranian diplomatic posts and severing diplomatic relations, but such moves were rejected at this time for several reasons.

One reason acknowledged by the State Department was that the remaining Iranian diplomatic personnel will be able to provide communication and assistance to the estimated 50,000 Iranian students in this country. The United States expects that help for the students will be the main function of the remaining diplomats.

The cutback in diplomatic personnel is expected to have little effect on the U.S. effort to free the hostages Messages for Tehran have only occassionally been passed through the Washington embassy.

State Department officials said most of the Iranian diplomatic personnel here are not needed under present circumstances. In any case, they added, the maintenance of such a large number of personnel in this country is utterly inappropriate in this situation.

Another reason for maintaining diplomatic relations, in the absence of actual diplomacy, is to avoid possible ill effect on the 50 American hostages at the U.S. Embassy in Tehran or the three U.S. diplomats being detained at the Foreign Ministry there.

"We would not have taken these steps if they would have endangered the hostages," Reston said.

Iran's charge d'affaires in Washington, Ali A. Agah, was notified of the cutback by Undersecretary of State David Newsom.

The reduction will not affect Iran's mission to the United Nations in New York. The chief of the cultural secton of the embassy here, Mansour Farhang, is expected to be the new Iranian contact man or formal emissary to U.N. Secretary General Kurt Waldheim, Farhang met in Qom last week with Ayatollah Ruhollah Khomeini and Foreign Minister Sadegh Ghotbzadeh. w

In another development, Mississippi River pilots have refused to navigate a ship headed for Iran with a load of soybean oil unless President Carter personally approves the shipment.

The refusal of the pilots prevented a Liberian registry freighter, the Jubilee Venture from heading past the treacherous bends of the river to open waters Tuesday. Capt. Charles Arnoult, head of the Crescent River Port Pilots Association, said he believed it to be a "patriotic duty" to block the shipment for Iran but "if the government overrules me on it, that's fine."

A spokesman for International Matex Co., which loaded the vessel, said the shipment was approved by the State Department.