An angry President Carter told House members last night that congressmen who consistently vote against him can expect retaliation from the White House.

"I'll be damned if I'll send my wife into your district for a fund-raiser," one congressman reported Carter as saying.

The statement came just hours after the House defeated legislation implementing the Panama Canal treaties, a vote that embarrasses Carter politically and casts some doubt on his administration's ability to keep commitments abroad.

"The biggest disappointment this year was the House vote on the conference report on the Panama Canal," Carter said at a White House dinner for the United Democrats of Congress, a group of about 60 moderate to conservative House Democrats.

Despite his emphasis on the treaties vote, House members were under the impression that he was referring to their voting record as a whole when threatening to deny White House favors, according to some of those present. They were also under the impression that the remarks were spawned by a whole series of recent congressional setbacks, including defeat of the budget resolution in the House, a large increase in defense spending in the Senate, problems with the oil windfall profits tax and other energy legislation.

The president's angry eruption came after one of the members had complained about access to the White House.

"Some of you people who complain vote with us only about 15 percent of the time," Carter was quoted as saying, "There are Republicans who vote with me a least 35 percent of the time."

Carter said those who vote against him could not expect favors. "We believe in rewarding our friends and punishing our enemies," he is reported to have said, adding, "We keep your voting records on computer in my desk drawer in the Oval Office."

"If it's grants for a city, then it is not proper to consider a congressman's voting record," he reportedly continued, but then said that in considering other requests and favors sought of the White House, a member's voting record would be scrutinized.

He said invitations to White House dinners, requests for campaign help, appointments for filling patronage jobs and other favors would be doled out on the basis of how a member voted.

"Some of you in Congress haven't supported me at all. I'll be damned if I'll send my wife into your district for a fund-raiser." Carter reportedly said.

Carter also said his new Cabinet secretaries at the departments of Transportation, Energy and Housing and Urban Development would be more "politically motivated" than their predecessors, indicating they too would look at voting records in passing out federal help.

Carter did single out at least one congressman he would help. He told House Budget Committee Chairman Rep. Robert Giaimo (D-Conn.), "Bob, if there's anything I can do for you, you'll get it."

One House member, who confirmed Carter's remarks, said it was "bad tactics. It's bad to have to resort to that kind of thing. I think it caused resentment rather than help."

Another House member said his colleagues were grumbling about the statement mentioning the computer list in the desk drawer as they rode a bus from the White House back to the Capitol.

In other remarks at the dinner, Carter also reportedly told the lawmakers he would not give up the quest for his programs or be frightened into not running for reelection. "Damn it, I'm a fighter," he was quoted as saying. "I'm tenacious, I don't give up."