U.N. Secretary General Kurt Waldheim has told the Carter administration and Egypt that United Nations peace-keeping forces will not police this week's first phase of Israel's withdrawal from the Sinai peninsula, U.N. sources said yesterday.

Waldheim, who met with Soviet Premier Alexei Kosygin and Foreign Minister Andrei Gromyko in Moscow last week, reportedly was told that the Soviet Union firmly intends to veto a renewal of the U.N. Emergency Force now in the Sinai, when the present U.N. mandate expires in July.

A Soviet veto would thwart administration hopes for using UNEF troops to police the Egyptian-Israeli peace treaty, which provides for a three-year staged withdrawal by Israel from the Sinai.

President Carter, who is expected to make a personal plea to Soviet leader Leonid Brezhnev at their summit meeting next month to drop the veto plan, has promised to organize an alternative multinational force outside the United Nations if UNEF cannot police the treaty.

Secretary of State Cyrus Vance instructed the State Department last week to say that the United States was not considering sending American troops to police the Sinai accord. State Department spokesman Hodding Carter declined to say what the administration would do if other nations do not controbute troops to an alternative force opposed by most Arab nations and the Soviet Union.

One idea being put forward by some diplomats at the United Nations and in Washington to avoid a Soviet U.S. impasse is to have a much smaller group of U.N. military truce observers replace the 4,000 member UNEF unit to watch over the treaty implementation. These diplomats hope that the Soviets would agree not to oppose the observers, whose presence would be less significant politically than the U.N. Emergency Force.

The emergency force was established by the Security Council in 1973 to watch over the disengagement of the Egyptian and Israeli armies after the October war. The seven nations contributing soldiers include Poland and Indonesia, which are highly unlikely to continue their participation in view of Soviet and Moslem opposition to the Egyptian-Israeli peace treaty.

Waldheim was reportedly sounded out by Washington and Cairo about the possibility of moving UNEF troops from their present positions in the Sinai to participate in the initial turnover of the town of El Arish this week and in subsequent Israeli withdrawals.

But linking the emergency force to the peace treaty would immdiately have brought a challenge in the Security Council, and Waldheim told the United States and Egypt he would be exeeding the original Mandate by assigning UNEF that function, according to U.N. sources.

Diplomats in New York said Egypt has been discussing the possibility of trying to preempt the scheduled July 24 vote on the present UNEF mandate by basing an appeal for an entirely new force on Security Council resolution 242 which followed the 1967 Arab-Israeli war and might be more acceptable politically to the Soviet Union that the 1973 Mandate.

U.S. officials report that Egypt and Israel have agreed to handle the El Arish return with their own forces and joint Egyptian-Israeli patrolling is one option under study for policing subsequent phases of the treaty.

The Israeli newspaper Ma 'ariv also reported on Sunday that a joint Egyptian-Israeli military committee has agreed that evacuation of the Sinai will be supervised by Egyptian and Israeli liason officers, who periodically will check the buffer zones that are to separate the two armies as the withdrawal proceeds.