A prospective deficit-reduction agreement between the Reagan administration and congressional leaders ran into a broad front of opposition on Capitol Hill yesterday, even as participants in the high-level negotiations struggled unsuccessfully to conclude an accord.

Meeting for the 18th day since the talks began in the wake of the Oct. 19 stock market collapse and with less than three days before $23 billion in across-the-board spending cuts will automatically occur if no accord is reached, negotiators said they were inching closer to a tentative agreement to cut the deficit by $30 billion this year.

But their progress was slowed by continuing disagreements over the mix of tax increases and spending cuts needed to achieve their second-year goal of reducing the deficit by about $45 billion in fiscal 1989, which begins next October.

As they worked behind closed doors at the Capitol, however, lawmakers of various ideological persuasions expressed grave reservations about the emerging plan and Wall Street analysts looked on with increasing dismay.

Republicans complained that the proposed agreement requires too little in spending cuts to justify the $9 billion in higher taxes, liberal Democrats criticized it for treating defense spending too generously, and moderates of both parties said the package simply cut the deficit too little to reassure troubled financial markets.

In the negotiations, White House officials sought to write in lower taxes, fewer defense cuts and higher domestic cuts for the second year of the plan. Then, as opposition to the impending agreement built among rank-and-file legislators, administration negotiators offered yet another proposal seeking further, similar changes in the first year, according to one source close to the talks.

A White House official said the administration was seeking an agreement on which specific taxes would be increased and on a mechanism for locking in the spending reductions for two years but were meeting Democratic resistance.

"The president gave in on defense and tax cuts and we'd like something in return," the official said.

When negotiators left off Tuesday night, the plan called for a total deficit reduction this year of $30.2 billion, with a $9 billion tax increase, a defense cut of $4.9 billion, discretionary domestic reductions of $2.6 billion and entitlement cuts of about $4 billion. Other savings would come from user fees, improved tax collections and asset sales.

An indication of the trouble facing the plan came late yesterday afternoon when Senate Minority Leader Robert J. Dole (R-Kan.) and House Minority Leader Robert H. Michel (R-Ill.) went to the White House to tell President Reagan that they are close to a deal, but one which is widely unpopular among Republicans and liberal Democrats.

"We may want to meet with him to tell him how bad it is," Dole had said earlier in the day. "It's pretty weak. A pretty weak package unless you like taxes."

A key Democrat in the talks, House Budget Committee Chairman William H. Gray III (D-Pa.), expressed similar sentiments. "If tomorrow morning we are still as close as we are, we perhaps should go to the White House and try to complete the negotiations there even if it takes a long time," said Gray. "There's two seconds left, we're down by two and it's time for the three-point play."

But other negotiators predicted, with varying degrees of confidence, that rank-and-file members of the House and Senate who have not been privy to the protracted and difficult negotiations would eventually come to support the emerging agreement once they realize the only alternative is the automatic cuts mandated by the revised Gramm-Rudman-Hollings budget-balancing law.

"I believe there will be a change once they know what's in it measured against the draconian cuts we face," said Sen. Pete V. Domenici (R-N.M.), the ranking Republican on the Senate Budget Committee.

A senior White House official made similar arguments. "It avoids the {automatic cuts}, it's the art of the possible," said the official. "It sends a message to markets that the White House and Congress can function."

But many members of both parties said they understood full well what was in it and what the alternative was and are still unconvinced.

"Seldom have so many people worked so hard and produced so little," said Rep. Willis D. Gradison Jr., a moderate Ohio Republican who serves on the budget and tax-writing committees. "They labored mightily and produced a mouse."

Gradison, whose views were apparently widespread among Republicans who caucused yesterday morning, said GOP House members think that "the president has sold out Republican principles by agreeing to tax increases in order to maximize defense spending and he went easy on his push for domestic spending cuts."

Other Republicans said many of the cuts were illusory and that they were leaning toward supporting either an alternative suggested by a bipartisan group of senators for a one-year freeze on all federal spending or favored letting the automatic cuts go into effect. Many say they would oppose plans by congressional leaders to postpone the automatic cuts scheduled to take effect Friday.

"It's not going to be a palliative to the markets, they're going to laugh," said Sen. Alan K. Simpson (R-Wyo.), the deputy GOP leader of the Senate. "I'm to the point where I'd rather take" the automatic cuts.

Liberal Democrats seemed equally disgruntled. Rep. Marty Russo (D-Ill.) said the package in its current form is "horrible" and "capitulates to the president's numbers again." The leadership, said Russo, "may have a tough time selling this in the House."

On Wall Street, analysts frowned on the disarray in Washington.

Robert Barbera, chief economist at E.F. Hutton & Co., derided the pending accord as "mealy-mouthed." Despite the 508-point drop in the Dow on Oct. 19, said Barbera, "We're going to get essentially what was being talked about before the crash."

Others voiced fears that a weak budget deal will undermine efforts by Treasury Secretary James A. Baker III to gain an international accord aimed at propping up the global economy that includes agreements by allies to stimulate their economies.

"I can't see how Baker is going to be in a very strong position to bargain unless something comes through on the budget that looks more attractive than what we're hearing," said Carol Leisenring, chief economist at CoreStates Financial Corp., a Philadelphia banking company.Staff writer Paul Blustein contributed to this report.