Ever since he bought almost one-third of District of Columbia National Bank last year, people have been calling Jeffrey N. Cohen "the boy banker."

When the 31-year-old Cohen revealed he was selling his D.C. National stock for a $1 million profit and was planning to bid for control of two other banks, the word was that "the boy banker" had learned his business in a hurry.

The plan, announced by Cohen earlier this month, was to buy controlling interest in Hemisphere National Bank and to make a public tender offer for all the stock of Bank of Columbia.

But now it looks as if part of Cohen's bank stock swap could be in trouble.

Minority shareholders of Hemisphere are marshaling opposition to Cohen's takeover, contending it would take control of the bank away from the Hispanic community that Hemisphere was chartered to serve.

On numbers alone, Cohen should have no trouble taking over Hemisphere. He has options to buy 59 percent of the shares, enough to elect a majority of board members.

But the Office of Comptroller of the Currency is not the Electoral College; it behaves more like Congress in making its decisions. Opposition by a vocal and creative minority can delay even a routine regulatory matter, perhaps even kill it, as Hemisphere officials learned this week.

A while back, Hemisphere applied to the comptroller for permission to open its first branch at 18th and Columbia Road in the heavily Hispanic Adams-Morgan neighborhood.

Branch applications are the bread and butter of bank regulation, but recently Hemisphere informed the Advisory Neighborhood Commission that it was dropping the application.

Details of the decision to back off are murky, though it's known that the comptroller's staff discussed the matter during a meeting this month with Hemisphere officials.

A formal objection to the branching bid was filed by the Food and Allied Services Trades Metro Washington Council, a labor group known as FAST. FAST's chairman is Tom McNutt, president of Local 400 of the United Food and Commercial Workers union, which is trying to negotiate a contract for Hemisphere's tellers.

FAST complained that Hemisphere had made few mortgage and home-improvement loans in the Adams-Morgan neighborhood. The ANC also put pressure on the bank, and neighborhood leaders served notice this week that they're opposed to letting Hemisphere slip out of Hispanic hands.

Cohen, ANC Chairman Emmanuel Lopez complained in a letter to the bank, "is hardly the type of person who will serve the best interests of our community."

Cohen said yesterday he is "aware there may be some concern that a non-Hispanic has an option to buy control" of the only bank now controlled by Washington's growing Latin community.

"I will give a lot of consideration to that point of view," added Cohen, suggesting that good management and a solid financial basis are more important to Hemisphere's ability to serve the community than Spanish surnames on the board of directors.

It will be next week before Cohen decides whether to exercise his option to buy the bank. He's in the midst of an inspection period that allows him to look at the books before buying, and he says he still is uncertain what he'll do.

Cohen said he was unaware there was opposition from minority shareholders to his buying the majority interest in the bank. The objectors include Eduardo Pena Jr., an attorney and former Hemisphere board member; Lucas Gallegos, a Northern Virgina businessman who's now on the bank's board; and a number of small stockholders.

"The whole purpose of chartering minority banks is lost," if the bank is sold to Cohen's group, Pena complained yesterday.

"We applied for this charter out of conviction that the Hispanic community was not getting a fair shake from the lending community." It took 2 1/2 years to get the bank charter and raise the capital to open for business, he added. "That will all be lost. We'll just be another bank."

Control of Hemisphere now is solidly in Hispanic hands. The biggest single investor is Nestor Julio Garcia, an Argentine banker who owns 28 percent. Chairman Leveo Sanchez owns almost 14 percent, and Director Angel S. Roubin has almost 10 percent.

Cohen has options, through a series of transactions, to buy all that stock plus an additional block of about 8 percent controlled by Peter Davis, president of Development Associates of which Sanchez is chairman.

Cohen's option agreement calls for him to pay about $3.15 million for the stock. With a grouop of as-yet-unidentified associates, Cohen plans to offer approximately $7 million for Bank of Columbia.

Cohen won't say who is putting up the funds for either purchase. Cohen himself appears to have a $1 million profit coming from his sale of 31 percent of D.C. National Bank stock that he bought for $40 a share last December and is selling for $60.

That quick profit could prove another stumbling block to Cohen's two takeover plans. Federal bank regulators traditionally frown on speculative trading in banks, believing that frequent changes in control of financial institutions aren't conducive to bank stability.