While President Reagan was vacationing in California after Christmas, his military planners were working around the clock pondering potential bombing targets in Libya ranging from antiaircraft batteries to oil refineries to terrorist training camps outside Tripoli, according to informed sources.

The problems confronting the Joint Chiefs of Staff in drafting military options for Reagan underscored the difficulties of planning a surgical strike that would spare civilians yet destroy targets directly linked to the Libyan-supported terrorists allegedly responsible for the Dec. 27 airport massacres in Rome and Vienna.

But the exercise also dramatized other non-military difficulties facing the Pentagon brass, some of which have popped up in other crises: a commander-in-chief vacationing 3,000 miles away and disinclined to signal his leanings on possible retaliation; a chief of staff and new national security affairs adviser also disinclined initially to get involved, and confusing rhetoric from the White House and State Department.

Adm. William J. Crowe Jr., the new chairman of the Joint Chiefs, felt obliged immediately following the airport attacks to prepare military options that would enable Reagan to fulfill his 1981 vow of "swift and effective retribution" for terrorist acts. Planners were summoned to pinpoint suitable bombing targets in Libya.

While specialists at the Central Intelligence Agency, Pentagon, White House and selected military commands studied maps and satellite photos, Reagan was in California without the chief advisers who usually steer the White House through such crises. Chief of staff Donald T. Regan and Vice Adm. John M. Poindexter, White House national security affairs adviser, did not join Reagan in California until Jan. 1 and 2, respectively. Defense Secretary Caspar W. Weinberger, sources said, did not step into the target planning at the outset.

In the absence of presidential guidance, the Joint Chiefs looked at every conceivable military target in Libya in case the president suddenly wanted to strike, sources said. The portrait drawn by Pentagon officials resembles students trying to study everything because they did not know what would be on the test; they also express frustration at what they sometimes view as an absentee president.

Pentagon planners also found themselves confronted with the same problems plaguing planners seeking to retaliate for the bombing of the Marines at the Beirut International Airport in 1983: intelligence sources could not pinpoint the location of terrorists responsible for the airport killings; civilians would likely be killed if the most attractive targets were picked; Americans abroad would face further retribution, this time from Libyan leader Muammar Qaddafi.

In Lebanon in 1983, intelligence officials said they had concluded that the only worthwhile military targets were Baalbek, near the Syrian border, and a radar center near Beirut. In Libya, the pickings were even slimmer. Bombing the oil centers would be the best way to hurt Qadaffi but would kill innocents, they said, while hitting the terrorist camps would link the retaliation to the crime but perhaps kill others who were not members of the Abu Nidal group that the State Department believes organized the airport shootings.

In the case of Lebanon, officials said, Weinberger and the Joint Chiefs, particularly then-chairman Gen. John W. Vessey Jr., were cool toward the proposal to strike Baalbek in concert with French bombers, partly for fear it would prompt more terrorist suicide attacks against the Marines. Robert C. McFarlane, then Reagan's national security affairs adviser, was known to favor the strike.

The French grew impatient at the U.S. ambivalence and struck Baalbek ineffectively on their own, rendering the target "cold" for a followup retaliatory strike by the United States. The United States did bomb the radar center on Dec. 4, 1983, as well as Syrian anti-aircraft sites in Lebanon, losing two bombers in the process. The painful memory of those losses made the chiefs cautious about Libyan targets, particularly the antiaircraft sites, sources said.

In sending their option paper to Reagan in California last week, sources said that, the chiefs recommended against bombing targets in Libya that were not linked directly to terrorists. Since his return to Washington, Reagan has indicated he will only approve a strike when the guilty parties are known and there is little danger of killing innocent bystanders, conditions which the intelligence community has not yet been able to guarantee, officials said.

However, government sources said, in the wake of the airport attacks Reagan neither discouraged the Pentagon from drafting strike plans nor guided the planners.

The Joint Chiefs, Pentagon officials said, had no way of knowing whether the president would order a strike "at first light" the next day, as occurred in the Dec. 4, 1983, raid, in which some bombers were forced to launch from their aircraft carrier before they were ready.

Also, harsh words from White House spokesman Larry Speakes about the possibility of military retaliation against Libya gave the impression inside and outside the Pentagon, officials said, that some prerequisites for a strike had been removed.

This situation accelerated the military's "what if" exercises and led to contingency plans to strike Libya with Air Force B52s based in North Dakota, Air Force F111 bombers in Britain, and Navy F/A18 bombers from the aircraft carrier Coral Sea in the Mediterranean.

Military leaders yesterday were still making military preparations under what the Defense Department called "prudent planning," including sending repair kits to the air base at Sigonella, Sicily, to allow the EA6B electronic warfare planes sent there last week to remain indefinitely if Reagan opts to increase the U.S. military presence in the Mediterranean.