A House-Senate conference committee, under renewed Reagan administration pressure to act this week, broke into sharp disagreement yesterday over a Democratic move to reduce the federal deficit that would be allowed this fiscal year in legislation designed to force a balanced budget by 1991.

By a mostly party-line vote of 36 to 12, House members of the committee adopted an amendment lowering the mandatory deficit target this year from the $180 billion set in the Senate-passed measure to $161 billion.

In response, an angry Sen. Bob Packwood (R-Ore.), the conference committee chairman, described the Democratic proposal as "nonsense" and accused Democrats of attempting "to absolutely choke a new process before it has a chance to work."

After a flurry of exchanges, Senate conferees left the meeting, promising to return today with a counteroffer that Packwood said later would probably set this year's deficit target at $171.9 billion, the level that in the congressional budget resolution adopted last August.

The dispute over the deficit target, the first major issue tackled by the conferees, revealed the continuing deep divisions over the deficit-reduction bill and threw into doubt optimistic predictions by congressional leaders that agreement on the measure can be reached this week. House Democratic aides said there remained a strong possibility that the conference committee will deadlock.

The committee is facing a rapidly approaching deadline of midnight Friday because the legislation includes an increase in the federal debt ceiling that will be needed by then.

Yesterday, President Reagan's budget director, James C. Miller III, increased the pressure by warning of widespread "pain and suffering" if the debt ceiling is not lifted this week.

The dispute over reducing the deficit target for this year was the first openly partisan battle in the maneuvering over the deficit-reduction legislation. House Democratic leaders have called the bill a "Republican incumbents protection bill" because it would not require votes for deep cuts this year by 22 Republican senators who are facing reelection.

"This takes away some of the protection," said Rep. Thomas J. Downey (D-N.Y.).

But Packwood complained that in appropriations bills Congress has already exceeded the budget resolution target of a $171.9 billion deficit for this year and that it would be foolish to lower the deficit target before deciding which federal programs would be subject to automatic cutbacks.

He said the Senate may counter with an offer to set the deficit target at $171.9 billion, but only if all federal programs are cut to the spending levels set in the budget resolution before any additional across-the-board cuts are imposed.

Republicans believe that such an approach would first force spending cuts in a number of domestic programs that they charge Democrats are seeking to protect.

But Democrats said that if the Republican-controlled Senate favored a mandatory process of deficit reduction it should also favor setting lower deficit targets in the bill.

"We should take the medicine now," said Rep. David R. Obey (D-Wis.). "We should cut the deficit now, when the economy can afford it."

As passed by the Senate, the balanced-budget legislation would set mandatory targets for the federal budget deficit, beginning at $180 billion this year and declining in $36 billion annual increments until the budget was balanced in 1991.

Under the measure, the president would be required to submit and Congress would be required to adopt yearly budgets meeting the deficit targets. However, if at the beginning of a fiscal year the projected deficit exceeded the target, the president would be required to impose across-the-board cuts in most federal programs except Social Security to bring the deficit within the target.

The conference committee, which met several times last week without resolving differences over details of the bill, yesterday resumed work in an atmosphere of growing pressure. The deficit-reduction measure is part of legislation to raise the national debt ceiling to more than $2 trillion.

Last week, the Treasury Department informed the conference committee that if the debt ceiling is not raised by Friday it would cancel government securities held by the Social Security trust fund and other government retirement systems. That would free additional government credit under the existing debt ceiling, allowing the government to continue to borrow and to function until about mid-November.

But it would also represent an extraordinary tampering with the politically sensitive Social Security system, costing it and other retirement systems interest. Resort to this is something that most members of Congress would like to avoid.

At the White House yesterday, Miller warned that if the congressional deadlock is not broken soon the government may be forced to shut down, causing "a lot of pain and suffering."

Miller said in a statement, "If some action is not forthcoming immediately, the president may be forced to take specific, responsible action to minimize the adverse impact of this inordinate delay."

Miller said Reagan had made no specific decisions on steps to be taken but added that "the government would have to close down."

Meanwhile, House Speaker Thomas P. (Tip) O'Neill Jr. (D-Mass.) continued to predict passage of deficit-reduction legislation this year while voicing objections to the Senate-passed measure.

O'Neill said the bill shifts too much power from Congress to the president, would weaken national defense, hurt the poor "very badly" and cause skyrocketing unemployment in the event of a recession.

But, O'Neill added, "There's no stopping it." He said about 75 House Democrats had informed constitutents that they supported the deficit-reduction legislation, effectively locking themselves in.

"It will be a big vote [for the bill] when it comes," O'Neill said. "The mail is overwhelming for it."

Rep. Les Aspin (D-Wis.), the chairman of the House Armed Services Committee and another sharp critic of the Senate bill, agreed with O'Neill's political analysis.

Aspin, who has warned that the budget cuts mandated by the measure would seriously weaken national defense, said some liberal Democrats support the bill because they want the Pentagon budget slashed. He said others in Congress think it would force Reagan to accept a tax increase, while still others think that it is so riddled with constitutional and other legal imperfections that the legislation would "self destruct" before it could take effect.

"Some also locked themselves in early in support of this and don't want to admit now that anything is wrong," Aspin said. "You can count the White House in this category."