Secretary of State Cyrus Vance wound up two weeks of intensive diplomatic negotiations on the Middle East today, reaching what American officials described as a substantial measure of agreement toward reconvening a Geneva peace conference.

High-level U.S. officials said today that by bringing the Soviet Union back into the Middle East picture, Vance attempted to provide "stimulus" to all parties involved.

Last week's joint Soviet-American communique on the Middle East provoked a furious reaction in Israel and among American supporters of Israel as well as among Americans who take a hard line toward the Soviet Union.

In meetings with Israeli Foreign Minister Moshe Dayan, also attended by President Carter, Vance outlined a new Israeli-U.S. approach to the meetings in Geneva.

This new Israeli - American formula has drawn what American sources described today as "constructive" reactions from the foreign ministers of Egypt, Jordan and Syria.

These three key participants in the proposed Geneva meeting, according to the sources have advised Vance that they may hold a joint meeting of foreign ministers later this month to formulate a response to the American-Israeli formula, if and when it gets formal approval from the Israeli Cabinet. Diplomats here believe that the Israelis will accept the formula but with reservations.

The shape of the still secret U.S.-Israeli position, as disclosed by Arab diplomatic sources here today, suggests a largely procedural, but important, maneuver to get around Israel's objection to the participation of the Palestine Liberation Organization (PLO) at Geneva.

The key aspects of the proposal have been relayed informally to the Arabs by Vance.

According to the Arab sources, the PLO issue would be by simplynot issuing formal invitations to Geneva. Instead, U.N. Secretary General Kurt Waldheim and the chairmanof the conference, the United States and the Soviet [WORDS ILLEGIBLE] issue a call to recovene the conference by urging all parties to participate.

In this fashion, according to the diplomats here the United States and Israel would preclude Security Council consideration of the issue and possible attempts to reformulate Security Council Resolution 242, which since 1967 has been a key basis for the Geneva talks and a Middle East settlement.

Washington Post correspondent H.D.S. Greenway reported from Jerusalem that the Israeli-American formula, according to reports in the Israeli press, calls for the opening session to include a joint Arab delegation including Palestinians.

The Israeli government today, however, refuted reports in the American press that it had agreed to the participation of the PLO itself at Geneva or to allowing the PLO to select Palestinians who might participate.

According to diplomats here, the Israelis are reported to have shown greater flexibility on the role the Palestinians should play at Geneva, reportedly going somewhat beyond the concept of a largely ceremonial role at the opening session between Israel and a unified Arab delegation.

All these are procedural matters. As one diplomatic observer put it, the U.S.: Israeli formula "is a device for starting the Geneva Conference, not a device for producing results."

The formula has yet to be considered and approved by the Israeli Cabinet, a step expected next Wednesday after Dayan returns to Jerusalem.

American efforts in the protracted and difficult diplomatic dealings this week revealed the Carter administration approched to the Middle East problem. By seeking to reach agreement on procederal matters, senior administration officals are convinced that they in fact are dealing with substantive issues.

"People sometimes make fun about debates and negotiations regarding the size of the shape of the table or whether it ought to be one table or two tables or who participates in what," a senior White House official said. "But these so-called procedural discussions really are substantive discussions because they mask, they undercover, the political reality which is determined by procedural agreement."

The most difficult procedural problems that remain are said to focus on the composition of various working groups that would discuss various aspects of the Middle East problem at Geneva. As one senior official put it, "When you define the working groups you are defining the agenda."

American officials insisted tonight that U.N. Resolutions 242 and 338 remain the basis for the Geneva negotiations. "By the time they go to Geneva," one American said, "the PLO would have to accept 242 even though they may state their reservations" about the lack of mention of "the rights of the Palestinian people."

American officials would not comment on yesterday's cautious PLO statement that suggested a measure of flexibility in the leadership of the guerrilla organization, but they seem to believe the Soviets have exerted a positive influence on the PLO.

Members of the American delegation to the United Nations went out of their way in private conversations to defend the administration's decision to involve the Soviet Union at this stage of the preparation for Geneva.

The argument is that the Soviets would have to be drawn into the picture at some point, that they could exercise either negative or positive influence and, as one American diplomat put, "We'd rather see them playing a positive role."

Egyptian and Jordanian sources are privately optimistic about the course of events. Jordanian Foreign Minister Hassan Ibrahim said in a speech before the General Assembly that the U.S.-Soviet statement on the Middle East was a "positive development."

Jordan "finds in the declaration a sound major step which could be the stepping stone for positive movement" toward Geneva, he said. Emerging from a meeting this morning with Vance, Ibrahim voiced optimism that the Geneva Conference would be reconvened before the end of the year. But he refused to discuss details.