The American decision to support an Argentine resolution calling for negotiation on the Falkland Islands was made at a Cabinet meeting on Monday, despite a personal appeal from British Prime Minister Margaret Thatcher to President Reagan that the United States abstain, American officials said today.

The conciliatory gesture toward Argentina -- which also constituted a rare public split with Britain -- came less than a month before the president's fence-mending visit to three Latin American countries -- Brazil, Colombia and Costa Rica.

In London, Foreign Minister Francis Pym called in the American ambassador today to express British dissatisfaction with the American decision on the resolution, which is to be voted on by the General Assembly Thursday or Friday.

British officials at the United Nations admitted that London was "dismayed and angry" over the move, and expressed concern that the American announcement would erode support by other nations for the British position against the resolution.

Already, Western diplomats said, three of Britain's nine partners in the European Community -- France, Ireland and Italy -- have indicated their intention to vote for the text put forward by Argentina and 19 other Latin American nations.

Thus far in the assembly debate, only New Zealand has fully backed the British stance against the resolution and against a resumption of negotiations with Argentina on sovereignty over the islands.

Assembly resolutions are not legally binding on governments, and British Ambassador Sir John Thomson has already served warning that the adoption of this one would "not alter the policy of the British government."

He said Britain opposed any U.N. resolution, and argued that if it entered talks on the basis of the present text, the outcome would be prejudged in Argentina's favor.

The British had won broad support at the United Nations after Argentina invaded the Falklands last April, but since the islands were recaptured by a British task force last June, the diplomatic tide has turned.

Argentina has sought to salve its national pride and shore up the domestic position of its government through a public manifestation of international support at the world body. To achieve this, it has radically modified the terms of the resolution, which is now expected to win an overwhelming majority.

American officials reported that after negotiations with Assistant Secretary of State Thomas O. Enders, Argentine Foreign Minister Juan Aguirre Lanari agreed to drop the "most offensive" aspects of the initial text, and submitted a final version that "we do not see as prejudging the position of either party."

The Americans said they had been able to remove a reference to pro-Argentine resolutions on the Falklands adopted by the Nonaligned Movement, and "the most offensive reference" to the British occupation of the islands as a form of colonialism.

In addition, American officials said, Argentina agreed to insert references to the "interests" of the 1,800 islanders, to the "de facto cessation of hostilities" and to the "expressed intentions of the parties not to renew them."

As it now stands, the officials said, "The United States does not see the resolution as setting a strict deadline for the start of negotiations" and it satisfies Washington's desire for a peaceful settlement of the dispute.

It was Aguirre Lanari who leaked the American voting decision to reporters at the United Nations yesterday, and U.S. officials here later confirmed it.

Thatcher'a appeal to Reagan, which outlined the reasons for the British stand, apparently reached the White House before Monday's Cabinet decision, and the president's reply explaining the American reasons for supporting the resolution was sent subsequently. There was no sign that the gesture to Argentina was strongly resisted by those in the State Department who favor closer ties with Britain, and one source commented that "it wasn't that tough a battle."