The nation's ailing banking system has been targeted this year for its most sweeping transformation since the crisis years of the 1930s.

This month the Bush administration plans to send to Congress proposals intended to combat the banking system's mounting financial losses and the erosion of the fund that insures bank deposits.

Not since the Depression years has the system been in such trouble. Federal regulators expect about 180 banks with a total of $70 billion in assets to fail this year, including at least one or two large banks, such as the Bank of New England, which has been close to failure for months.

Altogether, more than 1,200 institutions have failed since 1984, a major reason the number of commercial banks has dropped from more than 14,000 a few years ago to 12,399 at the end of the third quarter of 1990.

In the first nine months of 1990, about 1 in 10 banks in the country lost money. Ten out of the 48 U.S. banks with more than $10 billion in assets were unprofitable, according to the Federal Deposit Insurance Corp., which manages the bank insurance fund that protects depositors with up to $100,000 in an account.

The weakness of some of those bigger banks worries federal regulators most. Even two or three failures among them could exhaust the fund, raising the possibility that taxpayers would have to pay off depositors at failed institutions.

The key to improving the banking system's health and protecting taxpayers is finding ways to make banks more profitable and efficient, Bush administration officials have concluded. Doing so would require deep cuts in the industry's operating costs, most likely by slashing the number of banks through mergers and sales of weaker institutions. Regulators in some cases will force reluctant banks to merge, some administration officials believe.

Even the chief executives of some deeply distressed banks agree there are too many financial institutions. Said one chairman of such a bank in New Hampshire, "We are over-banked here, and some of us are going to disappear."

As the number of banks shrinks, thousands of bank employees will lose their jobs and the public likely will face higher interest rates on loans and higher charges for other banking services than they otherwise would have seen, banking analysts said.

"The ultimate protection for taxpayers is a strong banking system with adequate capital, and the banks have to be profitable to attract capital," declared Robert Glauber, undersecretary of Treasury for finance, who has the lead role in preparing the report to Congress.

In that report, the administration probably will propose settling the heated question of whether banks or their holding companies should be allowed to compete directly in the securities, insurance and real estate industries -- another approach to making banks more profitable. If they do, firms in those industries might be allowed to own banks.

Those and other recommendations are still being debated among senior officials at the Treasury and the White House, with various federal banking regulatory agencies and the Securities and Exchange Commission also involved.

The precarious condition of some medium-size banks and a few big money-center banks has forced President Bush to put the banking overhaul at the top of his list of domestic priorities for the coming year.

The banks' troubles are contributing directly to the U.S. economic downturn. Large losses have caused many institutions to cut back their lending. As a result of the banks' new conservatism, recent efforts by the Federal Reserve Board to cut interest rates and pump money into the banking system have given the economy less of a boost than they normally would.

Congressional leaders have agreed that some issues will have to be handled on an urgent basis, such as forcing banks to come up with as much as $25 billion in new money for the insurance fund, and reforming some aspects of federal deposit insurance.

An agreement is considered less likely on allowing banks to move into new areas of business, which has been strongly opposed by insurance and real estate interests. The securities industry is less opposed to the likely administration plan, so long as banks' securities activities are conducted by separate subsidiaries of a bank holding company and do not have access to funds from the holding company's federally insured banks, according to statements from the Securities Industry Association.

Entry into any of the new businesses, however, is not likely to be a panacea for the banks. Most institutions will have neither the expertise nor the money to get into the securities business, a business that is currently less profitable than banking, financial analysts said.

Collectively, U.S. banks earned $15.4 billion in the first three quarters of 1990. But they also wrote off as uncollectable $21.1 billion in bad loans, most of them involving risky real estate projects and debt-heavy corporate takeovers and mergers.

The banks with the largest losses also usually had the lowest profits, so that the losses had to be covered by taking money out of their capital. A bank's capital is the difference between the value of its assets, such as loans, investments and cash, and its liabilities, including what its depositors have in their accounts and other money the bank has borrowed.

The rising losses, tighter enforcement by some federal bank supervisors of long-standing regulations and new internationally agreed requirements for minimum levels of capital have squeezed many institutions to the point where they hardly want to make new loans.

When regulatory officials have tried to twist bankers' arms to encourage more lending, some bankers have protested that they simply can't raise new capital in the current economic and stock market environment -- the price of most publicly traded bank stocks dropped sharply in 1990. Instead, the bankers are holding down their loans. That works just as well for the banks, since it is the ratio of capital to loans and other assets that is important, not the absolute level of either.

But that does not work as well for the health of the U.S. economy, which needs credit in the form of bank loans to grow. Increased bank lending is a key step in the process of translating lower interest rates into more economic activity.

Chairman L. William Seidman of the FDIC has predicted that by the end of 1991, the fund will have only about $4 billion left to insure more than $2 trillion worth of deposits. The fund will lose $10 billion next year as a result of bank failures and declines in the value of assets taken over when banks have failed in the past, he projected.

Should the current slump turn out to be deep and long, rather than short and mild as the majority of forecasters now predict, then a larger number of institutions probably would fail and the insurance fund losses over the next three years would far surpass the worst-case estimate of $63 billion offered recently in a study prepared for the House Banking Committee.

The Treasury report, which is expected by the middle of this month, will seek repeal of a federal law that effectively bars interstate branch banking. Some 32 states now allow bank holding companies based anywhere in the country to own and operate banks in those states, according to Treasury officials. Most other states allow outside ownership of banks on a reciprocal basis within different regions, but some do not.

To make sure that banks do not abuse any of the new powers they would be given under the administration proposal, they will be required to provide separate capital with which to operate any new business.

Furthermore, funds from insured accounts in banks owned by the holding company could not be used in those businesses in any way. And as one more safeguard, poorly capitalized institutions would not be permitted to enter the new business, Treasury's Glauber said.