Secretary of State Edmund S. Muskie and a team of senior U.S. officials met Algerian intermediaries in daylong meetings yesterday in search of new approaches to the seemingly intractable, and increasingly public, negotiations over the 52 Americans held by Iran.

The length sessions at the State Department provided an authoritative account from four Algerian officials of the attitude of Tehran authorities about the details of a potential settlement, as well as a rare first-hand report on the condition of the hostages from two of the Algerians who met all of the captive Americans Thursday night and Friday.

Even as the diplomats were meeting, American television networks were broadcasting the third and final installment of the Christmas films of the hostages supplied by the Iranian government. Yesterday's films showed 12 hostages, which, according to State Department sources, brought to 41 the number seen in film footage or still pictures over the last three days. U.S. networks quoted Iranian authorities as saying other hostages did not wish to be photographed.

At the State Department, neither American officials nor the visiting Algerians had any substantive comment to make on the diplomatic discussions, which are expected to continue today.

The main purpose of the meetings, according to U.S. officials, was to acquaint the Algerians with several optional ways in which the Carter administration is prepared to respond to Iran's recent demands for $24 billion in assets and "guarantees" in return for release of the hostages.

Muskie has declared the Iranian demands to be "unreasonable" and beyond the powers of the president, but has withheld a formal response until it can be discussed with the Algerians. The main line of thinking at high levels of the administration is to make clear again, in as positive a fashion as possible, the actions which the United States is willing to take in connection with Iranian financial assets and claims, even while also making clear once again what the United States cannot and will not do.

The developments of the last several days, including the release of the Christmas films and a series of emotion-laden statements by high officials both here and in Iran, appear to have further diminished the already small chance for a breakthrough to release of the hostages.

About the best that can be accomplished in the next round of indirect diplomacy, in the view of American policy-makers, is to keep the process of indirect negotiations alive for the incoming Reagan administration.

Two of the Algerian go-betweens, Ambassador to the United States Redha Mauek and the chairman of the Algerian Central Bank, Sghir Mostefi, arrived here Friday and appeared at the State Department at 10 a.m. yesterday to begin the round of meetings with the U.S. team.

A little over an hour later, they were joined by Algerian Ambassador to Iran Abdel Karim Gheraieb and Mohammed Bel Hossein, an Algerian Foreign Ministry official, who had just arrived from Tehran after spending more than eight hours Thursday night and Friday with 49 American hostages at a secret location there. The Algerians had met separately with the three Americans being held hostage at the Iranian Foreign Ministry.

Deputy Secretary of State Warren Chirstopher and the U.S. team of diplomatic, financial and legal experts on Iran conferred with the Algerians for nearly 2-1/2 hours before a luncheon hosted by Muskie, who also participated in the talks. The discussions resumed for several more hours in midafternoon.

While the negotiators huddled over financial issues that have emerged as the crucial problems blocking release of the Americans, other State Department officials studied the latest television films for clues to the condition and treatment of the hostages.

The State Department asserted last Monday that some hostages have received inadequate attention for serious health problems and that some are reported to have been confined in prison. Officials said yesterday that these statements were based on solid information that has not been refuted by the television pictures.

Some of the hostages have complained of solitary confinement, according to officials. In this context, officials expressed puzzlement and concern that as many as 11 of the hostages were not shown in the recent films, despite the fact that some of these Americans had participated in earlier filmed ceremonies.

Despite the impression of many American television viewers that the hostages were photographed in their place of residence, State Department experts said the indications are strong that the hostages had been at the film site for only a brief time before they were photographed.

The apparent fascination of some of the hostages with the fruit shown in the televised scenes, to the extent that one hostage could be seen stuffing his pockets with fruit, was thought to be an indication that this was a novelty in their diets.

Among the small surprises for television viewers, and some officials, was the statement by one of the hostages that he had seen films on videotape. American movies, such as the Woody Allen film, "Annie Hall," are reported to have been made available for viewing for some hostages via videotape cassettes.

It was also surprising by some that the yellow ribbon symbol is in such currency among the hostages. The two women hostages, Kathryn Koob and Elizabeth Ann Swift, were wearing yellow ribbons in their hair, and a yellow ribbon was seen as a decoration on the Christmas tree. Hostage Leland Holland of Laurel, Md., in his filmed comments shown yesterday, thanked the Mount Vernon Presbyterian Church of Alexandra, Va., for displaying yellow ribbons on the church.

The telvision pictures, which dominated U.S. news media as Americans celebrated Christmas, brought the names and faces of the hostages into the nation's living rooms once again at a time or rekindled distress about the captivity of the Americans. The publication in Tehran a week ago of the official demand for $24 billion in assets and "guarantees" greatly intensified the belief that Iran is asking for illegitimate concessions that amount to ransom.

Iran took another step into the public arena yesterday when Prime Minister Mohammed Ali Rajai read what he said were excerpts from the most recent U.S. government response to Iran, which was delivered by Christopher and his group of experts to the Algerian intermediaries in Algiers on Dec. 4. The publication of the Iranian demands, and now of parts of the U.S. side of the recent dialogue, will only make it more difficult for the two sides to adjust their positions, in the view of American officials.

State Department sources said the United States continues to be willing to explore a means of placing Iranian governmental assets in some sort of escrow account, perhaps under the supervision of the Algerians or another party, while the final disposition of money disputes is decided by an international claims settlement commission. But the United States is unable simply to transfer the funds involved to the Algerian Central Bank, as demanded by Iran, officials said.

The United States is ready to freeze any American assets of the deposed shah and his family while Iran seeks legal claim to them in U.S. courts, but $10 billion in "guarantees" of such funds cannot be transferred to Algeria, officials said. U.S. officials said it is highly unlikely that any Pahlavi fortune approaching such a vast sum is in this country.