Frustrated by the failure of Prime Minister Margaret Thatcher's radical monetary policies to produce any signs of improvement in Britain's deteriorating economy, her 10-month-old Conservative government is divided over what to do next.

Faced with the almost unanimous judgment of economic analysts that the drastic steps already taken are unlikely to reduce inflation or revitalize the economy, hard-liners in Thatcher's government want her to go even further and impose on the country what one of them has called "unparalleled austerity." They want to shrink the money supply further, make bigger cuts in government spending and act tougher with trade unions seeking large pay raises.

"Their answer when what they're doing is not working," said one of the hardliners' critics inside Thatcher's Cabinet, "is to do more of it."

The hardliners' opponents in the Cabinet fear the government already is alienating too many voters less than year after being elected. They also fear that unnecessary confrontations with the unions, of which they believe the current six-week-old national steel strike is an example, could lead to the kind of labor upheaval that helped bring down the last Conservative and labor governments.

"It would be foolish to think we can ignore the social and political consequences of what we do," Sir Ian Gilmour, a close Thatcher aide, said in a speech last week.

"If we don't get it right this time, and I think the next 18 months are critical," said Thatcher's Employment Secretary James Prior, "then we risk losing control of the country to the extremists on the right or the left."

Public opinion polls show that voters have indeed become restive with Thatcher's economic policies. So far her initiatives have offset income tax cuts with higher sales taxes, pushed the inflation rate above 17 percent, increased the interest on mortgages and other loans to between 15 and 20 percent, and cut tens of thousands of jobs in the giant, government-owned steel and auto industries, adding to already growing unemployment.

The dissenters in Thatcher's cabinet still agree with the basic principles of her economic changes: strict control of the money supply, less government involvement in the economy, encouragement of private enterprise, and a more equitable legal and power balance between management and unions.

"It's the pace of the change rather than the nature of it that worries me," Prior explained. "You should not push too far, too fast when making radical change in the economy."

But the hard-liners and their supporters in the press believe that much more must be done quickly. The influential Conservative news magazine The Economist, said last week that Thatcher's government has lost its momentum and has failed particularly to make large enough cuts in government spending.

"There are signs of desperation setting in," The Economist declared. "Resisting such desperation involves the government in keeping not just its nerve but its political momentum. Alternately cursing and not-quite-cutting the public sector is not enough."

Prior was criticized by the Conservative press and Thatcher's chancellor of the Exchequer, Sir Goeffrey Howe, for convincing Thatcher to move only gradually in Parliament to chip away at the privileged legal position of British labor unions.

Howe said in a speech Saturday that "it would be fatal to Britain's chances" to restore balance in labor-management "if this government lost its nerve and neglected its clear duty to take in hand the necessary reform of the law."

Howe, Treasury Chief Secretary John Biffen, Industry Secretary Keith Joseph and Trade Secretary John Nott form with Thatcher the hard-line team that decides the government's economic policy.

The counterweight to them in the Cabinet is composed largely of Prior, Glimour and Agriculture Secretary Peter Walker. Along with Foreign Secretary Lord Carrington and Home Secretary William Whitelaw, they were closely identified with former Conservative prime minister Ted Heath and the Conservative Party's more liberal wing before Thatcher replaced Heath as party leader and then became prime minister.

They see themselves as traditional Conservatives while regarding Thatcher's economic hard-liners as politically inexperienced radicals whose policy-making is dominated by "dogma and doctrine."

The two camps in Thatcher's government had been wary of each from the start. The tension was manageable and reasonably good-natured until recently. At Christmas, with no labor trouble in sight and with a few foreign policy successes including the Rhodesia cease-fire agreement and Thatcher's highly successful visit to the United States, most of the Cabinet members were still reasonably optimistic about the future.

But the unexpectedly lengthy steel strike, the threat of other labor disruptions, the increasingly gloomy economic outlook, and the shrinking of policy options for brightening it, have since taken their toll. The two camps in the Cabinet have taken to sniping at each other in carefully worded public utterance and blunter words by each side's supporters in the press.

Prior, for example, was pilloried in the Conservative press last week for his "softly, softly" approach to legislation curbing union prerogatives. The Daily Telegraph, a sounding board for Thatcher government hard-liners, went so far as to to suggest that a "handful of ministerial resignations" would be better than having the Thatcher government "dissipate the country's resolve for firm action" against the unions.

Prior's friends elsewhere in the press rushed to praise his "political courage and sound judgment," to quote a laudatory profile of Prior in Sunday's Observer newspaper.

It is believed here Thatcher could not afford politically to purge Prior or other dissenters from the Cabinet. It would, among other consequences, make her opponents more eager to have her replaced as party leader and prime minister if her policies become too unpopular with voters before the Conservatives, who won a 43-seat majority in Parliament in May, have to face the voters again in 1984.

Thatcher has frequently said that it may take two years or more for her policies to have a positive impact. Then, with the help of expected large revenues from the peak years of British oil production in the North Sea during the mid-1980s, she hopes to win back enough voter support to be reelected and continue her drastic restructuring of the British economy.

But Gilmour pointedly observed that "while I agree we are embarked on a program that could take the best part of two Parliments, [up to 10 years] to carry through, I also note that between the first Parliament and the second the electorate will have its chance of a say."