Prime Minister Margaret Thatcher is creating an uncomfortable dilemma for the Reagan administration in her determination to win a complete military victory in the Falkland Islands and restore them to full British colonial administration, according to well-informed diplomatic sources here.

U.S. officials earlier were worried that Thatcher might suffer a serious military setback that could lead to her replacement by an unfriendly left-wing government in Britain. But now they are concerned about the implications of a decisive British victory and abandonment of negotiation of Argentina's sovereignty claim, according to the sources.

Reagan administration officials fear that a humiliating defeat for Argentina will sour American as well as British and European relations with much of Latin America for a long time to come, according to the sources, and that a major British military presence in the South Atlantic to protect the Falklands from future attacks by Argentina could be very expensive for Britain and reduce its contribution to NATO defenses in the North Atlantic.

"We have that concern, too," a senior Thatcher aide said today. "But we've had to face up to it. This government has very considerably hardened its position now that it has been forced to repossess the Falklands by military means."

During a series of diplomatic negotiations following Argentina's invasion of the Falklands on April 2, Thatcher was pressured by the Reagan administration into accepting compromises that would have allowed the islands to be administered by a group of other countries while Argentina's sovereignty claim was negotiated. But since these deals were rejected by the Argentine military government and British forces began their counterinvasion, Thatcher has returned to her original position.

"Our objective is to retake the Falklands," Thatcher told Parliament today. "They are British sovereign territory, and we wish to restore British administration."

She indicated that the British colonial governor would return to the Falklands and the executive and legislative councils of the islands' 1,800 inhabitants of British descent would be reconstituted as soon as possible after a British military victory. She said Britain would veto any United Nations resolution calling for a cease-fire before either the withdrawal or the surrender of the Argentine occupation forces.

Thatcher appeared to rule out any later deal with Argentina on sovereignty, repeating "the future will have to be discussed with the islanders" and adding, "I should be amazed if they were not now more hostile to Argentina than they were before."

Reflecting Thatcher's views, according to her aides, Defense Secretary John Nott said on British television last night, "We wouldn't have sent a task force of 26,000 military personnel 8,000 miles . . . only to arrive there, and suffer tragic losses of our men, to say, 'Thank you very much, we are now going to sit down with you and discuss sovereignty.' "

Prominent among the future alternatives for the Falklands now being considered by the Thatcher government, according to senior government sources, is granting the islands independence with security guaranteed by Britain and other countries--including the United States and some Latin American nations--perhaps under U.N. auspices.

American officials reportedly are skeptical about how this could be achieved if opposed by Argentina, but British officials said it would be consistent with what Britain has done for other colonies and with U.N. decolonization guidelines.

Tonight, in a BBC radio broadcast to the Falklands telling the islanders "you can now look forward to the early prospect of liberation and a return to your normal way of life," Nott said that if they wanted independence the Thatcher government would consider it.

"It would be a very small nation on its own," Nott said.

"It would have to depend on the goodwill of its neighbors and also on the security provided by the United Kingdom and maybe by other countries. But there is no reason why the Falkland Islands should not be independent."

Nott also promised that the British government would lengthen the runway at Stanley airport and improve air communications to guarantee supplies to the Falklands--steps the islanders had lobbied for here in vain in the past. "But of course, as I have said several times," Nott added, "we would want to come to a sensible and wise and peaceful accommodation with the countries of South America so that everybody lives in peace."

American officials feel this would require some satisfaction of Argentina's sovereignty claim, according to sources here. They said Washington expects Argentina to remain hostile to Britain and threatening toward the Falklands, requiring a large British garrison there that would tie up British troops, warships and combat aircraft.

The cruise liner Queen Elizabeth II has arrived at South Georgia Island, 800 miles east of the Falklands, with 3,500 Scots and Welsh guards and Gurkhas to man a British garrison after the Falklands are recaptured, according to defense sources here.

Some warships from the naval task force could be left at the Falklands and the Harrier jump jets with the task force could be replaced by faster, longer-range fighter-bombers using an extended Stanley airport runway.

Members of Parliament here are being told this would not cost much more than maintaining the military resources elsewhere and that the troops' experience in the climate of the Falklands would keep them prepared for NATO defense duty in northern Europe, according to political sources. But American officials reportedly would still be concerned about their distance from Europe and the need to replace in the North Atlantic British ships tied up in the South Atlantic.

"We are not being pressured by the U.S. administration about this in any way," a senior British source said. But this source acknowledged that both American and British officials "are concerned about relations between Western Europe and the U.S. with Latin America" and that such worries are being stressed by some American officials who he said "do not necessarily reflect the thinking at the top of the Reagan administration."

Other sources here said Washington appears reluctant "to put the arm" on the Thatcher government about this because it now has little diplomatic leverage.

Thatcher has strong support in Britain for what she is doing--as measured both by public opinion polls and the prevailing mood of a large all-party majority in Parliament--plus apparently widespread sympathy among Americans outside Washington, according to these sources.