The American Bar Association refused today to endorse tough new penalties for businesses that commit white-collar crimes.

Bowing to a new interest in criminal law by members of its corporate and antitrust sections, the ABA's policymaking House of Delegates rejected recommendations from its criminal justice section that would have forced companies convicted of white-collar crimes to repay victims' losses, pay double-damage fines, undergo court-supervised probation and tell victims by mail that they can recoup their losses.

The delegates also gave the corporate and antitrust lawyers the right to testify as official representatives of the organized bar before House and Senate committees on issues of white-collar crimes -- until now the exclusive domain of the criminal justice section, composed of law professors, judges, prosecutors and defense attorneys.

This added input from ABA sections heavily weighted in favor of business interests could have great impact on Congress' revision of the federal criminal code. A revision embodying many provisions supported by the criminal justice section passed the Senate last year but failed in the House. Both bodies are expected to try again this year.

Today's voting followed four days of negotiations seeking to resolve differences and prevent a public fight.

Except for the question of penalties, all the differences were settled. The criminal justice section backed off a proposal to make responsible corporate executives criminally liable if a company didn't comply with federal laws in areas such as security regulation or environmental protection.

The interest in criminal law in the ABA's antiturst and corporate, banking and business laws sections reflects the current emphasis among federal and state law enforcement authorities on prosecuting white-collar crime.

While the antitrust and corporate sections are dominated by big business attorneys -- the head of the corporate law section's subcommittee on criminal law is William Kennedy, chief counsel of General Electric Co. -- Lee Loevinger of Washington raised the specter of small businessmen as the primary target of tougher white-collar crime penalties.

He said most antitrust cases are brought against small businesses that often are guilty merely because they didn't realize they were breaking the law and that cannot afford to fight government prosecution the way the industrial giants do.

In a pointed reminder to the ABA delegates, Loevinger said these small businessmen make up the bulk of the clients of the ABA's 250,000 members.

Kennedy, the GE counsel, said the financial harm of a white-collar crime is too complex for a sentencing judge to calculate and should be determined by a separate civil trial. This, however, would involve added expense for the victims, something the criminal justice section's proposal tried to avoid.

Moreover, Harvey M. Applebaum of Washington, representing the antitrust section, said the stiffer penalties proposed by the criminal justice section are incompatible wtih present antitrust laws which allow for triple damages to victims who file a separate suit.

Georgetown University law professor William Greenhalgh argued that many of the criminal justice section's proposed penalties have been endorsed by the ABA for the past 10 years as alternatives to prison

The ABA also endorsed a streamlined, more open set of procedures for disciplining errant lawyers. President S. Shepherd Tate said these proposals which have to win approval from high courts in each state before they take effect, meet the organized bar's "uItimate responsibility to the public, our clients.'

The new rules would seat non-lawyers on disciplinary boards and open the proceedings to the public once a preliminary investigation shows prob ability of a misdeed. They would end rules now in effect in most states which keep disciplinary procedures private until the final step and which are tilted in favor of the accused lawyer by giving him as many as three levels of secret hearings before any discipline is recommended.