President Reagan welcomed the release of 39 Americans held hostage for 17 days as a "moment of joy" yesterday, but insisted that those who killed American servicemen in Beirut and El Salvador "must be held accountable" and promised that the United States would "fight back" against international terrorism.

Secretary of State George P. Shultz, appearing stern in a news conference following the president's televised address, said the United States would try to find the Lebanese hijackers of TWA Flight 847 who killed Navy diver Robert Dean Stethem and bring them "to justice along with the people conspiring with them."

Another senior administration official, however, said it may be virtually impossible to find the hijackers, whom he identified as belonging to the militant Shiite group known as Hezbollah, or Party of God.

Reagan and Shultz also demanded the return of seven other Americans who had been kidnaped in Beirut before the hijacking and are still being held captive in Lebanon. Officials said it was impossible to gain their release along with the 39 others, but noted that Syria is working in behalf of the United States to seek their freedom from Islamic extremists.

The tough talk from Reagan and Shultz appeared not to apply to direct military retaliation against factions in Lebanon who carried out the hijacking or those who held Americans hostage afterward. A senior administration official, asked by reporters about possible retaliation, said that "vengeance is not a satisfactory basis for policy."

"It isn't to say that retaliation doesn't deter; it does," the official said. "But it will be a combination of attack on fundamental infrastructure and the purposeful use of force where it can be done in an unambiguous, effective way that will follow."

Nonetheless, the release of the 39 hostages in Beirut ended for Reagan one of the most difficult problems of his presidency, one that forced him to confront his own rhetorical attacks on President Jimmy Carter during the Iranian hostage crisis and which threatened to paralyze his presidency in the way it did Carter's. Even before yesterday, the White House was preparing a campaign to turn the safe return of the hostages into political capital for the president.

In a nationally televised address shortly after the 39 hostages left on a flight from Damascus to Frankfurt, Reagan declared, "Terrorists, be on notice: we will fight back against you in Lebanon and elsewhere. We will fight back against your cowardly attacks on American citizens and property."

Shultz renewed his campaign against Beirut International Airport, calling it "a kind of safe haven for terrorists." He said it should be declared off-limits to international air traffic.

Speaking into an open microphone just before his address from the Oval Office yesterday, Reagan, referring to the violent new Sylvester Stallone film, said, "After seeing 'Rambo' last night, I know what to do next time this happens."

But instead of the "swift and effective retribution" against terrorism that Reagan promised in 1981, officials said yesterday the president's main response to the 17-day crisis would be a renewed effort to preempt such attacks and a campaign against international terrorism. Shultz said the U.S. response is a four-point policy: making no concessions to terrorists; improved intelligence gathering on terrorists and their organizations; "defensive" measures such as beefed-up security at airports and U.S. embassies; and taking "active defenses" that would impose "costs" on terrorists and help "preempt and interdict" their actions.

"What we really need to think about is imposing costs and looking to the future and preempting" terrorists, Shultz said.

The senior official said, "I would put the emphasis less on retaliation and more on the purposeful use of all U.S. resources, including force, in a consistent way to deal with the global problem." He said this must "transcend what has happened in Lebanon."

The White House, after an embarrassing false start in the early hours Saturday when it appeared that the hostages were being released, responded with unusual caution yesterday as the drama played out its final hours.

National security affairs adviser Robert C. McFarlane decided not to give a midday television interview with NBC because the Americans, though en route to Syria, were still in Lebanon. Reagan waited for about half an hour after the Air Force C141 carrying the hostages had left Damascus to deliver his four-minute address.

Reagan telephoned the plane shortly after takeoff from his study adjacent to the Oval Office. White House officials said he told the pilot, Air Force Maj. Bill Edwards, "I would appreciate your passing on to your passengers how proud I am of their strength and courage, and that I look forward to their being reunited with their families as soon as possible. Give the passengers my heartfelt thanks. Happy landings."

In their assessment of the crisis yesterday, White House officials said they had won release of the Americans chiefly by pressuring Shiite Moslem leader Nabih Berri to shift his strategy.

A senior official said the United States had persuaded Berri to give up his goal of seeking to use U.S. leverage over Israel to release the 735 Lebanese prisoners it held. He said the United States generated so much criticism inside Lebanon and outside it that "there wasn't anybody standing up and saying, 'Right on, Nabih Berri,' but a lot of people criticizing."

In his address, Reagan thanked Syria, Algeria, Saudi Arabia and King Hussein of Jordan for their support. But he made no mention of Berri, whom he had earlier identified as the key figure in Lebanon who could release the hostages. Asked why Berri was omitted, Shultz said, "He has portrayed himself in various roles" as a mediator and a party to the hostage-taking, and "we have to do some sorting out about Mr. Berri."

White House officials said again yesterday they had made no deal with Israel to free the Lebanese prisoners it holds. Said the administration official, "At no time, from the first day to the last, did we ever urge, cajole, suggest, directly or indirectly by any U.S. official to my knowledge, absolutely never any hint of it from the president, that they [the Israelis] alter their policy about no concessions or, in this case, releases, at any point . . . . "

White House spokesman Larry Speakes said Reagan hoped to meet with the freed hostages but that there were no plans for a White House ceremony.