ORLANDO, FLA., OCT. 23 -- The nation's bankers were anxious enough about their future before Comptroller of the Currency Robert L. Clarke took the podium here today at the annual convention of the American Bankers Association.

What the government's top regulator of national banks had to say did nothing to make them any more confident.

The image of the banking industry, Clarke declared, "is more tarnished today than it has been in a half century."

Polls show public confidence in the safety and soundness of the banking system at an all-time low, with 35 percent of Americans convinced the system is "unhealthy," Clarke said.

And this decline in public confidence in the banks could not come at a worse time for bankers, he emphasized, because Congress is about to tackle far-reaching banking reform legislation. Starting next year, the lawmakers will consider not only the biggest changes in the deposit insurance system since it was created, but they are also expected to completely rewrite the federal laws that determine what kinds of financial services banks can provide.

Clarke's warning was so blunt that ABA officials who saw an advance copy of his text reportedly had urged him to tone it down.

But in a session with a small group of bankers on Monday, Clarke recalled that two years ago, he told the ABA the bad real estate loans then devastating the savings and loan industry were going to become the banks' problem as well.

"Not very many" paid attention, he said.

Even today, most of the bankers attending their industry's annual meeting reject any comparison between the bank and thrift industries. "We are not facing the same slippery slope as the S&Ls," Donald G. Ogilvie, ABA executive vice president, assured his members.

Ogilvie and out-going ABA President Kelly Holthus stressed that banks have a much bigger cushion of capital to protect against the kind of losses on real estate loans that wiped out the S&L deposit insurance fund and are continuing to force thrifts into failure.

But some signs of unease in the banking industry are clear: Attendence at the ABA convention this year is down from a year ago by about 1,000 people to 4,500 -- barely half the number who attended a decade ago. Only two of the nation's biggest banks -- Chase Manhattan Corp. and Security Pacific Corp. -- sent delegations to the convention this year. Most of the ABA conventioneers are small-town bankers like Joe Neff, president of Dale State Bank of Dale, Ind.

To Neff, Clarke's warning of trouble ahead is "too gloomy" and does not reflect what's happening at Dale State and its two sister banks, which among them have assets of about $110 million. "Business is excellent," Neff said.

Veterans said the mood of the ABA gathering was down as much as the attendence was and the convention was less lavish than past events.

"Reminds me of a savings and loan convention," said banking consultant William Ferguson, president of Ferguson & Co., which has offices in Washington and Dallas.

Despite the obvious irritation expressed by many bankers at any comparisons with S&Ls, analogies between the two industries were drawn again and again during the four days of speeches, news conferences and workshops. "The major force that is affecting banking is speculative excesses. We've had too much of a good thing," said Alex Sheshunoff, whose Austin, Tex., firm tracks bank- and thrift-industry performance.

The same real estate speculation that killed so many thrifts is now causing trouble for banks, he added, predicting that it will be several more years before real estate markets hit bottom.

A similar assessment of the damage bad real estate loans caused was sounded by Thomas LaBrecque, who is about to become chairman of the board of Chase Manhattan Bank.

Real estate loan losses recently forced Chase to lay off 5,000 employees and set aside $650 million to cover future losses on real estate lending.

It will take five to seven years to work through the real estate troubles in the New York City suburbs, he said, and the problems are still spreading.

"You have to recognize this is not a normal cycle," he said. "What you've got here is a normal real estate cycle and, on top of that, an S&L crisis that created a tremendous overhang of properties."

LaBrecque said damages inflicted by real estate lending make it even more important that federal banking regulations be changed to allow banks to get into new kinds of business. Bankers and regulators agreed that one of the reasons banks loaned so much money to real estate developers is that there were few other good investment opportunities available to them.

LaBrecque said the bankers badly need the power to sell securities and insurance and to expand into other financial services.

Banker after banker made the case during the last three days for deregulation that would allow banks to compete in all parts of the country and to offer the same services as competing firms outside the banking industry.

Richard A. Kirk, the incoming ABA president and chairman of United Bank of Denver said, "The time is now" for Congress to pass legislation that will give banks additional powers.

But the warning sounded by Comptroller of the Currency Clarke did not go unheeded. "This is a difficult environment in Washington," one ABA official said.