A House Democratic Caucus panel, spurred by complaints that Congress had lost its ability to manage the government's fiscal affairs, yesterday recommended a dramatic revamping of the process the House uses each year to consider and adopt the federal budget.

At the same time, the caucus committee refused to change House rules to allow Budget Committee Chairman James R. Jones (D-Okla.) and his chief rival for the chairmanship, Rep. Leon E. Panetta (D-Calif.), to remain on the committee.

Lawmakers currently are limited to three two-year terms on the Budget Committee, and Jones and Panetta have exhausted their time.

In addition, the panel, in an action certain to anger House Republicans, voted to restrict the time allotted for televised speeches.

Conservative Republican activists have been using this hitherto unrestricted time to attack the Democrats before millions of cable television viewers.

All the proposed changes must be approved by the 252-member Democratic Caucus, scheduled to meet Dec. 3 to Dec. 5. They also would have to be approved by the full House, where Democrats will outnumber Republicans this year by 252 to 183.

The budget process proposals, if finally approved, would substantially revise the way the congressional budget act has worked.

The act itself was passed a decade ago to bring order to budgeting.

In the last few years, the budget process has become twisted in the political crossfire over Reagan administration efforts to push through its tax and spending cuts. As a result, spending bills often have been delayed until the final days of a Congress or wrapped into a massive last-minute spending package.

"This would take the demogoguing out of budgeting and put some reality into it," said Rep. David R. Obey (D-Wis.), who began pushing the changes six years ago. "I think members are fed up with the chaos."

The proposal would eliminate piecemeal consideration of tax and spending bills and wrap them, and a budget resolution setting broad spending goals, into one omnibus package that the House would have to consider by June 30 each year.

The proposal would set stringent time constraints on the passage of all 13 appropriations bills, and would lump them and all budget and needed revenue-raising legislation into one monumental measure.

In recent years, Congress has spent months passing budget resolutions which do not have the force of law until they are followed up by additional spending and tax bills.

Rep. Martin Frost (D-Tex.), head of the caucus Organization, Study and Review Committee that approved the changes, said yesterday the proposal is certain to be controversial within the Democratic Caucus and its prospects for passage are uncertain.

House Speaker Thomas P. (Tip) O'Neill Jr. (D-Mass.) and Majority Leader James C. Wright Jr. (D-Tex.) both have expressed reservations about the proposal, according to Democratic officials.

Yesterday's action on the budget committee chairmanship also is expected to be controversial in the caucus.

Jones and Panetta, who have been lobbying colleagues for months for the chairmanship, have said they will ask the caucus to waive the House rule that limits membership to three terms.

The three-term limitation was intended to prevent the Budget Committee from becoming too powerful and to ensure that many members of the House are exposed to its workings.

The chairmanship has become one of the most powerful and high-profile positions in the House Democratic leadership.

It is expected to remain so this year with President Reagan, flush from a huge reelection victory, expected to make another big push to cut spending in his fiscal 1986 budget.

If Jones and Panetta are unable to win a waiver, a major battle for the chairmanship is expected among several junior members of the committee, including Reps. William H. Gray III (D-Pa.), George Miller (D-Calif.), Les Aspin (D-Wis.), Butler Derrick (D-S.C.), Brian J. Donnelly (D-Mass.) and Mike Lowry (D-Wash.).

In other matters, the caucus panel voted to make the House Democratic whip an elected position beginning in 1986. The whip now is appointed by the speaker and majority leader and can use the position to get a leg up on his colleagues in moving up in the House hierarchy.