President Reagan yesterday ordered withdrawal of four sophisticated radar surveillance planes from Egypt by mid-week and return of the aircraft carrier Nimitz to its position off the Lebanese coast, adminstration officials said.

The orders came after representations at the White House, State Department and Pentagon that the administration sees the situation involving Libya as cooling off. Although White House officials said Reagan was following the situation yesterday, no more military assistance is expected.

The four Airborne Warning and Control System (AWACS) planes from Tinker Air Force Base in Oklahoma scanned skies over Egypt, Libya and Sudan last week looking for Libyan warplanes that might spearhead an attack expected on Sudan by Libya.

The crisis began almost two weeks ago, sources said, when Egypt told the United States that it had discovered a plan by Libyan leader Col. Muammar Qaddafi to assassinate Sudanese President Jaafar Nimeri and his top aides, take over the airport in the capital, Khartoum, and land combat troops there in U.S.-built C130 transports.

According to Egyptian intelligence reports sent to Washington, Qaddafi was counting heavily on assistance from dissidents inside Sudan.

Pentagon officials said yesterday that Egyptian President Hosni Mubarak put much more stock in the intelligence reports about the planned coup than did the U.S. intelligence community, including the CIA and the Defense Intelligence Agency.

They had expressed doubt that Qaddafi would go through with the plan, sources said.

Mubarak, counted by the Reagan administration as one of its staunchest friends in the Arab world, nevertheless insisted that Qaddafi might launch the attack and requested U.S. military assistance, the sources said.

They said Egypt asked for AWACS planes, which not only track whereabouts of other aircraft but could also guide Egyptian fighters to targets in the sky.

A request from Mubarak carries considerable weight because the United States is counting on use of Egyptian bases, particularly Ras Banas on the Red Sea, as launching pads for the new U.S. Rapid Deployment Force designed to react quickly to emergencies in the Persian Gulf area.

Mubarak's warnings about Libya gained credibility, sources said, when Qaddafi began to deploy bombers, fighter planes and transports on fields in Libya close to the Sudanese border.

One of the most worrisome signs, officials said yesterday, was the positioning of four Soviet-built TU22 Blinder bombers so that they could bomb Khartoum. But the most menacing action by the Libyan air force, sources said, was one night flight by MiG fighters along the Sudanese border but beyond Sudanese airspace.

On Jan. 1, the United States established the Central Command, a military command designed to focus on the Persian Gulf and send quick-reaction forces there in emergencies. Sources said Mubarak, citing the perceived Libyan threat, said he was asking for just such assistance and was counting on the Central Command to provide it.

If Reagan had declined to send U.S. military aid, Mubarak could have backed away from letting his country be used as RDF deployment, officials said.

Administration officials who specialize in the Mideast said they were not surprised that Mubarak has since chosen to distance himself in public statements from the U.S. assistance he received. They said his most important audience is Arab, not American.

The officials also said Mubarak has a stake in looking strong in Arab eyes and would not want to be perceived as intimidated by Qaddafi.

Military officials said the U.S. response was impressive. The AWACS crews managed to track aircraft in the region day and night and provide vital information about the military situation inside Libya.

The Nimitz came on station rather late in the situation, after steaming from its position off Beirut to conduct unrelated flight exercises off the Gulf of Sidra near Libya, then to a position off Egypt last week to lend additional support.

E2C Hawkeye early warning planes aboard the Nimitz stood ready to supplement the AWACS watch, officials said, but were not needed.

In giving the coup attempt a "low probability" of being executed, U.S. intelligence agencies turned out to be correct, officials said. But, as one put it last night, "With Quaddafi, you just never know, and the president didn't want to take a chance once Mubarak had asked for our help."