The Thatcher government came under renewed political attack here this week for spending much more than expected to modernize Britain's independent nuclear deterrent while being forced to economize on conventional defenses.

Government officials confirmed that Defense Secretary John Nott is nearing the end of negotiations with the United States to buy a more expensive Trident D5 missile system in place of the less advanced Trident C4 that Prime Minister Margaret Thatcher chose in 1980 to replace Britain's aging Polaris sub-launched nuclear deterrent.

This became known at the same time that the Royal Navy's flagship, the aircraft carrier Invincible, was being sold to Australia to raise money for other British defense needs. The Invincible, Britain's newest and largest carrier, is one of 20 ships that have been offered for sale in an economizing program Nott announced last year.

Politicians led by members of Thatcher's own Conservative Party, military experts including a former defense staff chief and newspapers from the left-of-center Guardian to the right-of-center Daily Mail criticized both the impending Trident deal and the Invincible sale. This revived the debate begun last year over whether Britain needs or can afford its own nuclear deterrent when economic constraints are shrinking conventional defenses despite increased arms spending.

Nott responded that Trident will consume a maximum of 3 percent of the defense budget in any one year, beginning in the late 1980s. The major impact of Trident's cost, currently estimated at more than $15 billion for the missiles and four new submarines, would be delayed further if the Thatcher Cabinet, as expected, approves the purchase of D5 rather than C4. The package was estimated to cost $11.5 billion when the switch to Trident submarines was announced in 1980.

The Reagan administration already has ordered the D5s for the U.S. Navy. The larger D5 missiles have a longer range and greater accuracy than the C4 missiles.

Both the increased total cost and its delayed impact if Thatcher chooses Trident D5 could make it easier to cancel the project if Thatcher's Conservatives are defeated in national elections due within two years. The opposition Labor Party is committed to canceling Trident and banning all nuclear weapons from Britain while the centrist electoral alliance of the Social Democratic and Liberal parties seeks a less expensive alternative.

Even some critics who accept a British nuclear deterrent contend that it does not require the pinpoint accuracy or the 6,000-mile range of Trident D5, which would extend from here well beyond Moscow.

Informed sources here said U.S. Defense Secretrary Casper W. Weinberger is considering ways to make a Trident D5 deal more acceptable to the British Parliament, which must approve Thatcher's final decision. They include keeping down its cost and offering British firms subcontracting work on missiles to be manufactured in the United States for both British and American submarines. The British already would make their own submarines and nuclear warheads.

To make this acceptable to Congress, which also must approve any deal, sources said the British are considering pledges of naval assistance outside the NATO alliance area. This could mean keeping on active duty or in a high state of inactive readiness British warships Nott had planned to mothball.

Weinberger also has backed Nott's shrinking of the Royal Navy's fleet, despite charges by both British and American admirals that it will impair the defense of NATO supply lines across the North Atlantic. British sources said Weinberger has accepted Nott's argument that Britain can carry out its antisubmarine warfare role with smaller, cheaper ships than, for example, the Invincible.

The Invincible is first of three new British carriers for helicopters and vertical-takeoff fighters specializing in hunting submarines. Nott said Britain will keep the other two when ready in 1983 and 1985.