UNITED NATIONS, SEPT. 25 -- The United States, the Soviet Union and the three other permanent members of the U.N. Security Council today authorized another round of U.N. diplomacy seeking an end to the seven-year-old war in the Persian Gulf before considering a U.S.-backed worldwide arms embargo against Iran.

Secretary of State George P. Shultz and the foreign ministers of the Soviet Union, China, France and Great Britain did not say how long U.N. Secretary General Javier Perez de Cuellar has been given to pursue further diplomatic efforts with Iran and Iraq.

The decision to pursue further talks represents a setback for U.S. efforts, as announced in repeated White House and State Department statements urging quick movement toward an arms embargo.

As late as yesterday afternoon, only several hours after Shultz hammered out the new approach in a meeting with Soviet Foreign Minister Eduard Shevardnadze, a statement from the White House in the name of President Reagan called on the Soviets to endorse an arms embargo "rather than delaying and seeking opportunities to expand their own influence at the expense of peace in the region."

Shultz, in a news conference shortly after the U.N. announcment, portrayed the new course as a slower-but-surer strategy that has the advantage of keeping all the veto-wielding permanent members working together.

Asked if he has given up on an arms embargo, Shultz replied, "On the contrary, the distinct possibility, in fact the probability, {is} that will happen unless there is a satisfactory response" to the new diplomatic efforts.

Another U.S. official explained the U.S. shift more succinctly, saying that "the votes aren't there" in the Security Council for the arms embargo.

The Soviet Union and China were described by U.S. sources as unwilling to back an arms embargo against Iran at this time. Britain and France were described as ready to vote for an arms embargo and other strong measures.

The United States, while favoring an arms embargo, was also described as "caught in the middle," trying to maintain the unity and potential effectiveness of the five nations working together.

Shultz and others sought to give an air of urgency to the forthcoming U.N. diplomatic efforts by noting that the joint announcement of the new efforts calls for "rapid" implementation of the Security Council's July 20 demand that Iran and Iraq stop fighting and return to the internationally recognized borders.

Reporters pointed out, however, that the U.N. order two months ago called for an "immediate" end to the war, indicating even greater speed.

U.S. officials said they anticipate a new effort by Perez de Cuellar that probably would entail a mission to the area by him or a representative. A spokesman for Perez de Cuellar, however, said no consideration is being given to another trip to the area which he visited earlier this month. The spokesman said the full 15-member Security Council probably would not meet to formally adopt a mandate for new peace efforts before Oct. 1.

Shultz said the Security Council will continue "to make what preparations may be needed" for the imposition of sanctions and an arms embargo while the new diplomatic effort goes ahead.

He said he is "probably more skeptical" than some others about the willingness of Iran to stop fighting and return to international borders, "but I'd like to be proven wrong." At another point, Shultz said the odds on Iran's acceptance were "probably" not as long as "a million to one" but he did not specify how long they are.

Iraq has said it will accept the July 20 U.N. resolution if Iran does so. Thus, the U.N. "enforcement measures" such as an arms embargo would be aimed only at Iran as the recalcitrant party.

Iraqi Deputy Prime Minister and Foreign Minister Tariq Aziz, in a U.N. speech late today, reiterated his country's acceptance of the July 20 resolution. But he attacked Iran's efforts to interpret it in its own fashion and called on the Security Council "to move forward unhesitatingly, without succumbing to any kind of maneuvers or blackmail."

Perez de Cuellar reported to the five permanent members, in a discussion over lunch at U.N. headquarters, on his talks this week with Iranian President Ali Khamenei and his aides. U.N. sources said Iran offered a slightly different formulation of a de facto cease-fire that they had proposed earlier to begin simultaneously with the appointment of a U.N. commisson to assess blame for the war.

U.S. officials said the Iranian plan makes no mention of a withdrawal of forces to international boundaries. Shultz said repeatedly today that such a withdrawal is an essential element in the U.N. peace plan.

The luncheon at which today's decisions were made was the first working meeting of the foreign ministers of the U.N.'s "big five" since 1972.

Shevardnadze said as he left the meeting, "We vigorously support and approve the mission of the secretary general" as decided today. The Soviet minister did not say explicitly whether Moscow would support an arms embargo if the new diplomacy should fail.