While President Reagan struggled successfully last week to maintain U.S. unity and his prestige in the face of the Mideast airplane hijacking, his administration's widely heralded antiterrorist policy became a casualty of the hostage crisis.

The big loser was Secretary of State George P. Shultz, who for the past eight months has been telling any audience willing to listen that the United States "must be willing to use military force" to combat terrorism.

On Oct. 25, 1984, Shultz kicked off the antiterrorism campaign in a speech at Park Avenue Synagogue in Manhattan in which he said, "The public must understand before the fact that there is potential for loss of life of some of our fighting men and the loss of life of some innocent people."

A few weeks later, Reagan reportedly approved a covert operation directing the Central Intelligence Agency to train counterterrorist units for strikes against suspected Mideast terrorists, a policy favored by Shultz and national security affairs adviser Robert C. McFarlane.

This policy blew up March 8 along with a car bomb in a Beirut suburb that killed 80 persons and wounded 200. The bomb was directed at Mohammed Hussein Fadlalah, leader of the Hezbollah (Party of God), a militant Shiite movement that at this writing has custody of five or six of the passengers taken off Trans World Airlines Flight 847.

Fadlalah survived the explosion 50 yards from his home, although several of his bodyguards reportedly died in the blast. Fadlalah has been tied by U.S. intelligence to several attacks at U.S. facilities in the Mideast, including the Oct. 23, 1983, suicide bomb attack on the Marine compound in Beirut that killed 241 U.S. servicemen.

The attack on Fadlalah was directed by a runaway group of Lebanese and others. Although the bombing was not authorized by the CIA, the implications of the incident were so alarming for U.S. policy that Reagan disbanded the covert support operation.

Nonetheless, McFarlane continued to insist in subsequent speeches that the United States has the will and the ability to act against terrorist groups. Shultz and McFarlane are firm allies on this issue, as on many others. Both were architects of the ill-fated U.S. policy in Lebanon, and both insist on the need for antiterrorist operations, in part because of the reluctance of Defense Secretary Caspar W. Weinberger to use conventional military force against terrorist groups.

It is one of the internal ironies of the Reagan administration that the "conservative" Weinberger has been most realistic on the limitations of combating terrorism while "moderates" Shultz and McFarlane have been most insistent on pushing a policy of retaliation that in emotional appeal makes up for what it lacks in probability of success.

One of McFarlane's persistent advocacies is that the administration needs improved intelligence in the Middle East, an implied rebuke to William J. Casey, director of central intelligence. Casey has his shortcomings, but it is probably a romantic view to believe that the CIA is likely to have much success in penetrating militant Shiite groups in which a recruit may be ordered to carry out an assassination to prove his fidelity.

More realistically, some national security officials say that the United States lost much of its intelligence capability with the Israeli invasion of Lebanon on June 6, 1982, and the subsequent expulsion from the country of the Palestinian Liberation Organization. The PLO, for all its terrorism, was far more susceptible to U.S. penetration than the militant Shiite factions now trying to fill the Lebanese vacuum.

So far, the real moderate in the administration has proven to be the president, who somewhat plaintively expressed his frustration at the news conference Tuesday when he said he had "pounded a few walls" and added: " . . . You have to be able to pinpoint the enemy. You can't just start shooting without having someone in your gunsight."

The question now is how long Reagan can sustain this statesmanlike approach in the face of demands from prominent advisers to "do something" about terrorism.

Reaganism of the Week: When Reagan met Tunisian President Habib Bourguiba on Tuesday, he was asked if he would like the title of "president for life" that Bourguiba was given for leading Tunisia to independence. "I almost was the first two months I was here," said Reagan in reference to the March 30, 1981, attempt on his life.