The major American oil firms doing business in Libya indicated yesterday that they will bow to President Reagan's wishes and bring home any U.S. employes who wish to leave Libya. But the firms also said they intend to replace the Americans with foreign technicians and continue their Libyan operations.

In meetings at the State Department, the administration emphasized to representatives of some of the affected companies that it wants the 1,500 U.S. citizens in Libya to leave and is prepared to use tough measures, including possible federal court sanctions, against any who try to stay behind.

In its call Thursday for Americans to depart, the administration went out of its way to stress that it was not seeking to disrupt Libyan oil production. Instead, the move was presented as a "protective" measure aimed at getting U.S. citizens out of what many contend could become another Iran-hostage-type crisis if tensions between the United States and Libya continue to rise.

The companies, either publicly or privately, took the position that they will provide transportation and other necessary assistance to bring out those employes wishing to leave Libya, but most stopped short of saying that they would order their American personnel out.

In light of the U.S. government's position and the companies' willingness to cooperate, however, administration officials said they thought the vast majority--if not all--of the Americans would return home.

Senior administration officials continued their refusal to say whether the United States plans to follow up the departure of the Americans with economic reprisals or other steps designed to deter Libyan leader Muammar Qaddafi from supporting international terrorism and acting against U.S. interests.

Instead, the administration's posture was described by Treasury Secretary Donald T. Regan, who said in a television interview that it is up to Qaddafi to decide whether he wants to ease or exacerbate his dispute with the United States.

"I think there are steps that Mr. Qaddafi could take to ease the situation," Regan said. He refused, though, to specify what these might be or to discuss allegations that Qaddafi has dispatched assassination squads to murder top-level U.S. officials.

But Secretary of State Alexander M. Haig Jr., speaking at a news conference in Brussels, said Washington's purpose in trying to put Libya off limits to U.S. citizens was a response to "the very special problem" of terrorism "targeted on American officials" that had been disclosed by "a host of sophisticated intelligence-collection capabilities."

Haig would not describe these "capabilities," but he is believed to have meant electronic eavesdropping from specially equipped aircraft and other means in the vicinity of Libya. After his news conference, he and other officials in his party made clear that his remarks were intended to refer to the possibility of an actual terrorist attack against a U.S. official.

His comments appeared to tie Reagan's call for Americans to leave Libya much more directly to the "hit-squad" allegations than administration officials here have been willing to do. In part, Haig seemed to be trying to distinguish the U.S. position from that of America's North Atlantic Treaty Organization allies who are taking no parallel steps and who in some cases are notably cool about the U.S. approach to dealing with Libya.

Disclosure of the differing European views, becoming apparent at the NATO meeting in Brussels, continued yesterday when West German officials said that Foreign Minister Hans-Dietrich Genscher had told Haig efforts should be made to treat Libya in more friendly fashion. French Foreign Minister Claude Cheysson expressed a similar position Thursday.

The attitude of the Europeans, who are much more dependent on Libyan oil imports than is the United States, is understood to be a principal factor inhibiting Washington from declaring an economic boycott or taking other actions against Libya. Without the cooperation of the Europeans and other major Western oil-importing countries, economic sanctions will not have any notable impact on Qaddafi.

Some senior officials here privately expressed hope that the Europeans can be induced to change their minds. But Haig said Thursday that most are likely to make "no change whatsoever" in their policies toward Libya; deputy White House spokesman Larry Speakes pointedly noted yesterday, "What Haig said is the definitive answer."

Officials of the principal U.S. oil companies with Libyan operations--Mobil, Marathon, Occidental, Conoco and Amerada Hess--said they had been told by the Libyan government that their American personnel would be allowed to leave. However, the companies noted that moving the employes, together with their dependents and belongings, involves considerable logistical problems and will take at least a month or possibly longer.

Although the State Department said the companies were being cooperative, they were understood to be unhappy about the problems they will encounter in finding qualified replacements for the Americans and their ability to perform their contractual obligations to Libya in the interim. Some said they anticipated disruptions in their Libyan production that could last six months to a year.