The 20 House Democrats working to win support for a controversial tax-increase bill were unworried as they sat in the Capitol office counting votes Wednesday afternoon. It appeared they had the 218 votes needed to defeat a Republican procedural challenge to the deficit-reduction measure, which was scheduled for a vote the next day.

Then House Speaker Jim Wright (D-Tex.) entered and told the group two things. He had agreed, he said, to let an appropriations panel add $3 billion to a defense bill. And there was substantial opposition to including a welfare package in the bill, and he would permit procedures to strip it out if necessary.

Wright's remarks about those seemingly unrelated issues set off a chain of events that led Thursday to a bruising legislative defeat for the speaker, followed by a rowdy, vituperative floor fight and a one-vote victory wrung amid Republican charges of vote manipulation. It was perhaps Wright's toughest moment during his 10 months as speaker and has left its mark as the day the House went out of control.

"I think this has created tremendous damage with the Republicans versus the Democrats," said one House Democrat. "And there is damage within our caucus. A number of members are upset with the leadership."

Wright accomplished his objective of passing the tax bill, to give the House leverage in high-level budget negotiations among House members, senators and officials of the Reagan administration. Wright and other supporters said the bill would help assure financial markets looking for signs that Congress could rein in the federal deficit.

But members from both parties said the cost of the bill's passage could be the reverse: an end to the bipartisan tone Wright and others consider vital to the budget negotiations and another confused signal to Wall Street.

"He's not the speaker of the Democratic Party, he's the speaker of the House," said Rep. Lynn M. Martin (R-Ill.). "After he spends a week telling the country about bipartisanship . . . this sends the wrong signal to America."

Even those who are critical of Wright's handling of the tax legislation admit he was caught between conflicting forces. Democratic factions were pushing Wright to take actions that would enrage other factions.

Wright said after the final 206-to-205 vote approving the tax bill that it was "not a strong mandate." Yesterday, he said he hoped no bitterness remained from the floor fight, in which members of both parties abandoned decorum to an extent rarely seen in a legislature.

"I hope there is none," Wright said. "There shouldn't be . . . . This is a body of strong-willed people, and highly competitive people."

The sorest point for Republicans Thursday came when they thought they had defeated the tax bill on final passage by a 206-to-205 vote. Wright held the vote open for nearly 10 minutes after the official period to cast votes had ended, until one Democrat switched his vote to reverse the outcome. Yesterday, the office of the House parliamentarian said Wright acted in compliance with parliamentary rules.

Republicans demonstrated that some pique remained yesterday by forcing a recorded vote during what was to have been a pro forma House session. Because only 155 members were present, not enough to make up a quorum, the House was forced to adjourn.

Further fallout from the Thursday debacle is possible next week, when the House takes up a giant $597 billion bill to fund the government for all of 1988. The measure would be controversial in any case, but legislators said opposition may be sharpened as a result of this week's feuding.

Wright's decision to allow $3 billion in defense spending to be added to that appropriations bill, coupled with the deletion of the welfare provisions, induced Rep. Marty Russo (D-Ill.), the head of the Democratic whip organization on the tax bill, and other liberals to vote against the measure on final passage.

"How can we turn around in one full swoop and add $3 billion when we have been savaging the domestic side? I can't tolerate that," Russo said.

In the end, Wright brought the tax bill to the floor apparently without knowing whether there were enough votes to pass it. As he delayed announcing the tally, Wright tapped his gavel lightly, looking for someone to switch.

Sophomore Rep. Jim Chapman (D-Tex.) responded, rushing to the chamber from his office, having seen the losing vote on House television.

Chapman, who won his seat in a special election with strong backing from Wright and is a member of the Democratic Steering and Policy Committee, changed his vote from nay to aye, and the tax bill passed. He could not be the swing vote to torpedo a deficit-reduction plan, even though he disliked its specifics, he said later.