The largest stockholder of Financial General Bankshares Inc. boycotted the company's annual meeting yesterday, and with other shareholders accused the company's president of usurping control over nearly one-third of Financial General's shares.

Eugene Casey sat outside the meeting room and refused to vote his approximately 11 percent of the shares in an attempt to keep the company from getting a quorum for its meeting.

The meeting was held anyway. The quorum count repeatedly was challenged by attorney Samuel Jones Jr., whose firm represents Bert Lance and four Arab investors who own about 20 percent of the shares.

Along with Casey, Jones challenged the power of Financial General President G. William Middledorf to vote proxies over large blocks of stock owned by Cassey, and Arabs and others. Without those proxies, the company did not have a majority of share-holders represented at the meeting, they claimed.

Financial General attorneys defended Middendorf's claim to the proxies and said that, even without them, they had enough votes. "Assuming everything adverse to the position of management was taken out, we still had as quorum," said company counsel Martin Thaler after several hours poring over proxy papers to get a precise count.

The dispute over the quorum over-shadowed any substantive action at the meeting. A resolution authorizing the company to change its name to First American Bankshares was passed, and the company's slate of directors was elected without opposition.

Among those elected to the board were Casey, a veteran member who was renominated by Middendorf, and his son Eugene S. Casey, a new member chosen in an apparent attempt to make peace betwen the two factions.

Casey gave proxies controlling most of his massive block of stock to Middendorf last year when Middendorf was assembling a group of investors to take control of the company. Middendorf also got proxies from several other investors, many of whom since have sold their shares to the Arabs, Middendorf contends all the proxies are irrevocable.

From outside the door, Casey told reporters Middendorf "is in three using my proxy illegally. I'm not going to waste my money taking him to court; he's already spending too much of my money on lawyers."

During the meeting, Chairman B. Francis Saul said Financial General has spent "several hundred thousand dollars - at least $150,000 of it over budget" - on legal fees in the first quarter.

Financial General is awaiting a decision in U.S. District Court in Washington on a complicated lawsuit seeking to block an alleged takeover attempt by Lance, the Arabs and others.

Jones said it was "ridiculous" for Middendorf to claim be still controlled stock that had been sold to the Arabs. The proxies given Middendorf couldn't be revoked by the persons who gave the proxies, but they were not binding on people who later bought the stock, he contended.

Jones said he owns more than 100 shares of Financial General. He is a law partner of Eugene Metzger, who is involved with Lance and the Arab investors in the attempt to gain control of the company. Metzger's proxy is also claimed by Middendorf.

If Middendorf's voting of the stock went unchallenged now, lawyers said, it could reinforce subsequent claims when more than the quorium of a routine meeting is at stake.

In settling an SEC complaint accusing them of illegally seeking control of the company, the four Arab investors agreed to make a public tender offer for stock of Financial General. At that point, the disputed proxies could swing control of the company.

Casey, who has a reputation for being contankerous, railed at some Financial General officers, but never met Middendorf face to face.

Casey contends Financial General should be disbanded as a bank holding company, and its 13 member banks cut loose. The holding company contributes nothing to the affiliate banks and lives off their profits, he argued yesterday.

Saul, however, said he believes the banks are worth far more as a group, particularly because Financial General is the only bank holding company with affiliatesin the District of Columbia, Maryland and Virginia.