House Democrats yesterday abandoned for now an effort to curb after-hours televised speeches, a rules change directed at a group of conservative Republicans. The Republicans were opposed to it, and it turned out that many Democrats were as well.

Democrats, who yesterday ended a closed, three-day party caucus, said efforts at compromise failed and the proposal had been sent back to the caucus study committee that had recommended it.

"The mood is one of confusion and lack of consensus," said Rep. William V. (Bill) Alexander Jr. (D-Ark.). "We have not been able to come up with an agreement on how to prevent the abuse we have seen here this year while protecting the right of members to speak out on issues that interest them."

In other matters yesterday, the Democrats rejected, 176 to 53, a proposal to overhaul House procedures for handling the annual budget resolution and related spending and tax bills.

The proposal on after-hours debate, which is carried live to millions of cable-television viewers, would have changed House rules to limit it daily to two hours, divided evenly between Democrats and Republicans.

There is no limit now on such debate, called special orders. And a small group of conservative Republicans has spent hours after each session scoring rhetorical points against the Democrats. Few Democrats have shown up to defend their party or to take on the GOP, and Democrats have complained that the Republican offensive has hurt.

The caucus committee said such after-hours speech-making costs $10,000 an hour in overtime and other expenses. Any rules change adopted by the Democrats would have to go to the full House for final approval, but that approval would be essentially pro forma since Democrats will control the new House, 253 to 182.

During debate Tuesday, several liberal Democrats charged that the proposed rules would amount to a violation of freedom of speech. In addition, they said, it would play into the Republicans' hands by allowing them to complain about Democratic "gag" rules and by making it look as if the Democrats were afraid of the issues the Republicans are raising.

Democratic officials said the issue will be studied more and possibly brought up again in January.