West European leaders are about to launch their own diplomatic "fact-finding" mission in the Middle East after ruling out, for now, any direct challenge to the Carter administration's peace efforts there.

While both the exact form and intent of such a mission are expected to occupy discussion among European heads of government into Friday, Europe's chiefs were clearly in favor of renewed and expanded contact with the Arab world on possible further steps toward a solution to the Palestinian problems.

At the same time, their spokesmen sought to see American Concern that any European move in the Middle East might undercut President Carter's Camp David peace process.

As heads of the nine Common Market countries began two days of talks here, they stressed there would be no European-sponsored effort soon in the United Nations to supplement Security Council Resolution 242, which is the current framework for Egyptian-Israeli peace talks, nor would the Europeans formally recognize the Palestine Liberation Organization now.

But government spokesmen stressed "Europe's responsibility" to be prepared to step directly into the peace process if U.S. efforts run out of momentum -- which Europeans believe could easily happen by the end of this year.

"The Mideast is much too near to us to tell us to stay out. It is in our interest to explore solutions," said the spokesman for one major European power who asked not to be identified.

For this reason, the European leaders are expected to reactivate formal high-level talks with the Arab League, broken off in 1978 after Egypt joined in the Camp David peace negotiations and was thrown out of the Arab League.

Although the European-Arab dialogue has not been completely dormant since then, the Europeans now want to step up contact considerably.

Just how the renewed contact should be made was to be a prime subject of late-night dinner conversation. One proposal is for the head of the European Council to meet directly with the Arab League.

But in September, PLO chief Yasser Arafat is to take over as Arab Leagues president and high-level European contact with Arafat in whatever context would likely irritate both Israel and Washington.

Moreover, Luxembourg soon assumes the presidency of the European Council, and there is a feeling in Europe's major capitals that resumption of the dialogue with the Arabs should involve weightier European representation.

Consequently, another proposal being floated is for the Europeans to send a committee of high-level emissaries to Arab capitals.

The point of all this, say European government spokesman, would be to take "diplomatic soundings" of Arab positions on Palestinian question, although precisely what the Europeans could ask or find out that Washington has not, remains unclear and not publicly stated.

As Europe sees it, the Middle East conflict is not only one of the chief causes of developments leading up to troubles in Iran and Afghanistan; it is also a crucial obstacle to a constructive solution of the problems of the nonaligned nations and it prevents Israel from playing an important international balance of power role.

Any European approach to the Middle East, however, could disturb Israel which strongly opposes diplomatic recognition or contacts with the PLO.

Carter worries that anything that upsets the Israelis too much will ruin his effort to revive stalled negotiations on autonomy for Palestinians in Israeli-occupied territory in Gaza and the West Bank of the Jordan River. iCarter also has to worry about Jewish voters.

But the Europeans have been under pressure from oil-producing Arab nations in the Middle East, as well as Jordan, to involve themselves diplomatically in the Middle East.

The European leaders are expected Friday to issue a declaration separte from the usual joint summit communique, putting forth their position on the Middle East. Government spokesmen here emphasize the declaration is not likely to include anything new, but rather will be a re-statement of position foreshadowed in previous European Statements.

Sources said, however, the declaration will mention the PLO, and any such reference in a document signed by European heads of government would significantly edge, Europe toward at least de-facto formal recognition of the PLO.

The early part of today's talks involved a discussion among Europe's leaders of world economic problems, with and eye toward next week's world economic summit.