President Reagan ordered a freeze on hundreds of millions of dollars in Libyan assets in American banks and their subsidiaries yesterday, further tightening economic sanctions against Libya for the recent terrorist attacks in Rome and Vienna.

Reagan signed an executive order freezing the assets after U.S. officials discovered that Libyan money was beginning to drain away from American banks in the wake of the president's announcement Tuesday night of an economic boycott against Libya.

A senior administration official told reporters that the frozen Libyan assets amounted to "hundreds of millions of dollars," but could not be more precise. The official said Reagan moved as a "precaution" in case Libya seized the property of U.S. companies and citizens there.

But other sources said there is no evidence yet that Libyan leader Muammar Qaddafi has threatened U.S. property in his country. Rather, the sources said, Libya apparently was trying to move its assets out of the reach of the United States, which also froze Iran's assets in 1979 in response to the U.S. Embassy hostage crisis.

Economic experts said yesterday that Reagan's announced measures against Libya are likely to have only a limited effect on Qaddafi. Meanwhile, the Treasury Department followed up on the president's economic sanctions with regulations warning American citizens to leave Libya promptly or face up to 10 years in prison. And U.S. firms doing business with Libya said they would comply with his order but were uncertain of its impact on them.

As Reagan signed the new order on Libyan assets yesterday, U.S. ambassadors in Western Europe and the Middle East made renewed appeals to allied governments to join the economic boycott in retaliation for the Dec. 27 airport attacks in Rome and Vienna that killed 19 people, including five Americans.

Administration officials said they were considering a mission by Deputy Secretary of State John C. Whitehead to Western Europe and Japan to urge cooperation in the boycott, but a decision had not yet been made to send him.

Richard W. Murphy, assistant secretary of state for Mideast affairs, met yesterday with ambassadors of Arab countries in what U.S. officials said was an effort to persuade them that the campaign against Qaddafi is in their inter- est.

Reagan acknowledged again yesterday that it will be difficult for economically troubled Western European allies to join in the U.S. boycott of Libya. He said the allies were "torn" by his request for cooperation because their economies have lagged behind that of the United States.

In an interview with bureau chiefs from independent television networks, Reagan said the Western Europeans must weigh the costs of a boycott against the expense of "having to stay constantly on guard" at their airports and "a loss of tourist trade." Reagan said allied cooperation for a "short duration" could have the effect of forcing Libya to "change its ways."

While Reagan has hinted at possible military retaliation, he indicated yesterday it would probably not be ordered soon, saying he was "glad" there is some public support for a strike "if it ever becomes necessary." He cautioned, "We must not get tempted into creating a terrorist act on our own in response to a terrorist act."

Asked about a Libyan statement that the U.S. economic boycott amounted to a "declaration of war," he said, "I think if it ever came to a declaration of war, they'd be aware of the difference between what I said last night."

The president reiterated that "we do have the evidence" that Qaddafi is supporting international terrorism. He cited the seizure of Tunisian passports from two gunmen in the latest attacks that he said had been given to them by the Libyan government after it confiscated them from Tunisian workers being expelled.

The administration yesterday released a "white paper" describing 58 incidents -- most of them previously reported -- dating back to 1979 in which there was alleged Libyan government involvement in terrorist acts. Qaddafi "has used terrorism as one of the primary instruments of his foreign policy and supports radical groups which use terrorist tactics," the report said.

A State Department official, who spoke to reporters on condition that he not be named, said there is "no smoking gun" directly linking Qaddafi to the Dec. 27 airport attacks, but rather "circumstantial" evidence.

In his news conference, Reagan linked the recent attacks to Palestinian guerrilla Abu Nidal and his organization, which he said has been supplied by Libya. Reagan said the United States knew the location of terrorist training camps in Libya and that Abu Nidal had moved his headquarters there.

But the official said yesterday that while the administration believes Abu Nidal is in Libya, "we can't state that unequivocally."

The official said there are no training camps like an American military base, but rather various places, some of them temporary, for practicing and planning terrorism. The official said that the U.S. count of such training centers varies from day to day.

Presidential spokesman Larry Speakes said yesterday that "if Qaddafi strikes again and Americans are involved, the United States will be prepared to hunt down and to take drastic action against those responsible for it."

On the Libyan assets freeze, the senior official who briefed reporters said the property was not being seized from Libya, but Reagan's order would prevent the withdrawal or transfer of the assets from U.S. banks or their subsidiaries overseas. The action is allowed under emergency powers granted the president to deal with threats to national security.

The official said the freeze is expected to last "for some time." Also yesterday, officials were planning to use Voice of America radio broadcasts to urge Americans to leave Libya. I130:Picture, State Depertment spokesman Bernard Kalb: "It would be a very serious mistake to misread our determination. . . ." The Washington Post