U.S. Secretary of State Alexander M. Haig Jr. returned to the presidential palace here yesterday for a third day of talks with Argentina's government leaders in what appeared to be a tenacious effort to soften a so far inflexible Argentine claim of sovereignty over the Falkland Islands.

Haig continued talks until early this morning, meeting alternately with Argentine President Leopoldo Galtieri and Foreign Minister Nicanor Costa Mendez for a total of 12 hours in the presidential palace.

Haig, who reportedly obtained an extension of the talks yesterday by promising new proposals to an increasingly resistant Argentine leadership, appeared to be attempting to bring strong pressure on the military government in a marathon effort to end the confrontation between Argentina and Britain over the South Atlantic islands.

Officials on both sides of the talks indicated late yesterday, however, that the U.S. effort to arrange an accord that would prevent a military clash between its two allies appeared close to breaking down for the second time in 11 days of shuttle diplomacy.

By late yesterday afternoon,, U.S. officials said that the U.S. delegation's departure from Buenos Aires still had not been scheduled and that it had not been determined whether Haig would return to Washington or fly once again to London.

As Haig's hard bargaining began to produce signs of hostility among Argentine officials, as well as accusations in the media here of U.S. partiality to Britain, Costa Mendez said yesterday that there had been "no certain progress" in the talks. Other Argentine officials said privately that although the government had made concessions during the weekend, it could not offer more without endangering its own survival.

The key obstacle to a diplomatic accord between Argentina and Britain, officials indicated, remained Argentina's insistence on--and Britain's refusal to accept--an implicit recognition in any agreement of Argentina's sovereignty over the islands, which it seized April 2 in an armed invasion.

In his first public statement since arriving here Thursday night, Haig stressed to reporters yesterday that the U.S. position was that "the question of sovereignty, it seems to me, is best not raised in the context of the current crisis."

"There are different views on both sides," Haig said during a brief photo session. "Clearly it is a subject" for "negotiations later."

Haig conceded that the sovereignty issue "clearly has an impact on the whole conduct of what we are discussing," and diplomatic sources indicated that Haig had extended his visit precisely because he believed a softening of the Argentine position was essential to an accord.

High-ranking Argentine officials gave no indication yesterday that they were willing to moderate their stand. Galtieri, also breaking several days of silence, said yesterday afternoon that the islands, which Argentina calls the Malvinas, are "Argentine territory and we are not going to abandon them."

During the past two weeks Argentina's sovereignty over the islands has become the basis of the Galtieri government's fragile internal political support. "Sovereignty is not negotiable" is almost a slogan for the military government and the media.

U.S. officials, after failing to win Argentine approval for interim arrangements for the island's government Saturday, worked until early yesterday morning on language that would treat the competing claims to the isolated archipelago in a vague manner acceptable to both Buenos Aires and London.

"There was a real blizzard of paper up there," said one source of the hotel floor occupied by the U.S. delegation, which includes Assistant Secretary of State Thomas O. Enders, and several lower ranking regional and economic specialists.

Reports here yesterday said that Argentina had agreed over the weekend to allow British participation in the administration of the islands while negotiations proceeded. But the reports, which could not be confirmed, said that Argentina was still insisting on retaining the governorship in any temporary administration and delegating lesser posts to U.S. and British officials.

A British television network reported that Haig proposed that Britain and Argentina postpone the question of sovereignty for up to five years and name a nine-nation group to administer the Falklands in the meantime, Reuter reported from London. The report could not be confirmed.

Reuter also said that Prime Minister Margaret Thatcher returned from her country residence to London Sunday and will make a statement in Parliament on the Falklands early this week. She had briefly interrupted her weekend at Chequers on Saturday for a crisis meeting in London and returned to the country residence Saturday night.

As Haig appeared to step up the pressure on the Galtieri government for a concession on the sovereignty issue, Argentine government officials indicated yesterday that they might drop their cooperation with the U.S. mediation effort and instead seek diplomatic support in the Organization of American States.

Accusations of U.S. partiality for Britain began surfacing yesterday in Buenos Aires newspapers, attributed to military sources, and government-controlled television stations began broadcasting explanations of how Argentina could seek to invoke the inter-American treaty, or Rio Treaty. That accord calls for consultation among 21 American countries, including the United States, in the event of outside aggression.

Haig repeatedly has urged the Argentine government not to move to invoke the treaty and said yesterday, "There are a number of complications that perhaps would raise questions as to whether it was appropriate to invoke the Rio Treaty."

As a signatory to the pact, the United States theoretically would be bound by any Argentine move to set up a multilateral American defense against the British fleet that is slowly approaching the Falklands. Ten thousand Argentine troops reportedly are defending the islands.

Haig reportedly warned the Argentine military junta on Saturday that the United States also was bound by defense treaties with Britain and yesterday told reporters that the "complications" with the Rio Treaty came "in the context of our longstanding obligations to Great Britain, which are well-known."

Diplomatic sources here said that the Argentines' revival of discussion of the treaty indicated the military's defensive reaction to U.S. pressure during the past three days. Officials said, however, that Argentina is not considered to have the 14 votes necessary to mandate action by the 21 Rio Treaty signatories in the OAS.

In addition to stressing U.S. defense ties to Britain, Haig has attempted to maintain pressure during the talks by refusing to establish a firm schedule and by repeatedly signaling his willingness to make an abrupt departure.

On Saturday, Haig ordered the luggage of the U.S. delegation loaded on his plane and sent part of his party to the airport before announcing that he would stay.

Argentine officials gave no indication on movements by the Argentine naval forces that reportedly left port Thursday and headed south along the Argentine coast, which is about 400 miles from the islands.

Meanwhile, government officials announced that three of the six foreign journalists held in southern Argentina since Thursday on suspicion of espionage have been released. An American and two Canadians working for Canadian Broadcasting Corp. were released in the southern port city of Comodoro Rivadavia after being detained Saturday for filming military installations. Three British journalists remained in detention.