PANAMA CITY, JULY 5 -- Four weeks of intermittent violence have disturbed the international banking community here and, some bankers say, could threaten Panama's status as a leading financial center.

Some 125 banks, with assets estimated at more than $30 billion, are operating here, making Panama the most important center of American banking in Latin America and the Caribbean. It ranks ahead of the Bahamas and the Cayman Islands.

"Panama has been a convenient place for international banks," said a financial expert, "but it is by no means indispensable. If trouble continues here for much longer, bankers will think twice about staying on."

No one is predicting that a mass exodus is imminent from a country with nearly ideal conditions for a center of international finance: no taxes on bank deposits, no reserve requirements for foreign operations, no income tax on profits, and no currency conversion problems because Panama's unit of exchange is the U.S. dollar.

But when mobs chanting anti-American slogans hurled stones and bags of paint against the buildings of Chase Manhattan, Citibank and the Bank of America, alarm bells went off in corporate headquarters from New York to Tokyo.

A day after the mob incidents, workers were busy boarding up the plate glass windows of a string of banks in the financial district, giving visible expression to fears that Panama's troubles were far from over.

"There is a lot of nervousness in the banking community," said an international businessman. "A lot of people are 'reviewing operations.' "

Trouble flared after public accusations that Panama's military strongman, Gen. Manuel Noriega, had been involved in crimes from murder and drug trafficking to vote-rigging.

The charges, made by Noriega's former second-in-command of the powerful armed forces, Col. Roberto Diaz Herrera, set off a series of violent clashes between antiNoriega protesters and security forces.

On the first day of the protests, June 9, several international banks received anonymous bomb threats. Although no explosives were found, the warnings created a climate of uncertainty that continues.

The initial clashes followed a familiar pattern -- police vs. demonstrators. But since then, there have been a number of incidents that caused considerably more concern to many business executives.

On June 30, a crowd of 500 government supporters yelling "Yankee go home" stoned the U.S. Embassy and Consulate in protest against a resolution by the U.S. Senate urging a return to democracy in Panama.

The country has a civilian president who is little more than a figurehead. Noriega, who commands the 20,000-strong police and military forces here, is Panama's de facto ruler.

Following the embassy attack, blamed by Washington on Noriega, a third layer of violence was added to the already heated atmosphere here. Armed civilians attacked commercial establishments and offices belonging to opposition leaders.

A group armed with submachine guns stormed an exclusive clothing store and Jaguar car dealership and set fire to the building and several cars. Police stationed nearby did nothing to stop the vandals.

With many banks' profits already stagnating, some economic experts say continued insecurity could easily tip the balance toward a decision to scale down or pull out.

Latin America's recession and the continuing debt crisis have reduced overall business since 1982, when bank assets began plummeting from their 1981 total of $46 billion. Although the decline was halted in 1985, several major banks postponed plans for expansion.

Banking and financial services employ 9,000 people and account for around 8 percent of gross national product here, an important share of an economy that largely depends on income from the Panama Canal and the 10,000 U.S. soldiers stationed here.