Egyptian Foreign Minister Ismail Fahmi yesterday hinted that the Arab nations and the Palestinian Liberation Organization are flexible on finding a formula that would allow the PLO to participate in an Arab-Israeli peace conference.

At a news briefing following extensive talks with President Carter and Secretary of State Cyrus R. Vance, Fahmi restated in strong terms the Arab's willingness to accept the existence of Israel as part of a Mideast settlement.

"What's new in the situation in that the Arab countries are ready for the first time to accept Israel as a Middle Eastern country to live in peace in this area in secure borders. But these borders must be the international borders of 1967," Fahmi said.

"Security is needed not only for Israel, but it is needed more and more for the Arab countries, because of the continuing aggression of Israel . . . and because of the quantity of arms they have in their arsenal," he said.

Fahmi was asked whether Arab mayors on the Israeli-occupied West Bank of the Jordan River might represent Palestinians at a Geneva peace conference - a proposal made by Israel.

"I am not aware of the so-called assumptions that the West Bank mayors are any different from the PLO. I want you to be sure, they are the same," he said. "Let the Palestinians decide whom they would like to choose to represent them."

He said he was discussing a formula with U.S. officials that would be acceptable to all parties. "I am hopeful we can overcome certain difficulties. But we should not jump to conclusions," he said.

If the formula could be found, he predicted the PLO would accept U.N. Resolution 242, passed after the 1967 Middle East War, which in effect recognizes Israel's right to exist.

Israeli Foreign Minister Moshe Dayan told reporters after extensive talks here he thought "an agreed formula" could be found on seating Palestinians at Geneva. But he rejected participation by the PLO "as such."

Fahmi rejected separate negotiations or a separate agreement between Egypt and Israel.

On the whole, he gave the appearance of optimism over his discussions here.

He said there was unanimity now that the Geneva conference shoul reconvene this year.

There was agreement that the Palestinian problem should be dealt with as a political problem and not as a refugee problem, and consensus that Palestinians must be represented at Geneva, he said.

"Now we are discussing ways and means of getting to Geneva."

He said if Geneva conference did not materialize this year, it could take place early the following year.

Fahmi accused Israel of trying to slow progress toward peace settlement by engaging in "naked aggression" in Southern Lebanon.

Warning of a "dangerous, explosive situation," he said Israel would take the blame if fighting escalates further. "It is Israel aggression, naked aggression, anarchy," he told reporters.

Amid reports that Israel has seized strategic hilltops in Lebanon, the State Department yesterday renewed its call for restraint by all parties concerned.

"It is imperative that the fighting not spread in Southern Lebanon. We are making intensive efforts with all concerned to prevent that and to stabilize the situation there. We continue to urge restraint on all," a spokesman said.

Fahmi told reporters: "They [the Israelis] must have chosen to do this now to filibuster all efforts towards a peaceful solution and to getting to Geneva."