Challenger met chairman yesterday in a tense confrontation over age, experience and financial performance at the shareholders' meeting of Columbia First Federal Savings & Loan Association.

Harold N. Goldsmith, 45, who is trying to unseat three directors of the Washington thrift, went head-to-head with Chairman Dewitt T. Hartwell, 63, at the annual meeting at the Hotel Washington.

Goldsmith, who contends that Columbia First's financial results and management have been unspectacular in recent years, has been campaigning to get himself and two colleagues elected to Columbia First's board to influence the thrift's direction.

Goldsmith owns 10 percent of Columbia First's stock.

Preliminary results of the vote counting for the board seats are expected Friday.

Neither Hartwell nor Goldsmith, a Baltimore businessman who is chairman of the Towson-based retail chain Merry-Go-Round Enterprises Inc., would predict the outcome.

The atmosphere at the meeting, judging by applause and statements of support from the stockholders present, appeared to favor management.

But several stockholders said they voted for Goldsmith because they were impressed with his views.

"I wanted to inject a little youth and more vigor into the board," said Nathan Halpert of Rockville, a retired businessman, who said he voted his 1,000 shares for Goldsmith.

Stockholder Sidney J. Brown, president of Beltway Plaza Developers, who owns 9 percent of Columbia First stock, said he abstained from voting because he liked Goldsmith's experience but didn't want to hurt management.

The need for new voices on the board was at the heart of Goldsmith's campaign.

"This board is not made up of bad people," Goldsmith told stockholders. "But I think they need fresh thinking." Goldsmith said that while he respects age -- the youngest member of the current board is 55 -- younger people "can offer new vision and they can offer new ideas."

Hartwell responded, "I don't think there's a thing wrong with age, because sometimes with age comes wisdom." Discussing management's record, Hartwell added, "We've done a lot of things right."

Goldsmith, who also is president of Eastern Savings Bank of Timonium, Md., compared the record of his institution with that of Columbia First.

Goldsmith noted that Columbia First lost $33.5 million from 1981 through 1984. The thrift made money from 1985 through 1987. At Eastern, he said, "We had a profit in each and every one of those seven years."

Hartwell said Columbia's losses were because of the high interest rate environment and that, despite the problems, "We survived."

Goldsmith and Hartwell went at each other on several other issues.

Hartwell suggested that Goldsmith would have a potential conflict if elected to the Columbia First board because of Goldsmith's investment and involvement in Eastern Savings and other financial institutions in the Washington area. If Columbia First decided to expand, Hartwell said, "There's got to be a conflict."

Goldsmith, who has denied any potential conflict, did not discuss Hartwell's statements.

Goldsmith criticized Hartwell for suddenly deciding to drop the provision of the Columbia First charter that limited investors to buying 10 percent of the stock.

Hartwell announced the change two weeks ago after winning the support of Diana Corp., which owns 10 percent of the thrift's stock. Diana said it would support management in two elections, and in return, the thrift said it would back Diana's wish to buy up to 24.9 percent of the stock.

"Do you think that without the threat of losing three board seats that Mr. Hartwell would have recommended the change in the charter?" Goldsmith asked.

Hartwell said he had offered Goldsmith a seat on the board, but Goldsmith said the offer came with a nine-page list of "strings," including a demand that Goldsmith promise full support to Hartwell and the board's "goals and strategies."

Goldsmith said, "If I fully supported strategies that produced $16.7 million in losses over the last seven years, I wouldn't be standing here today."

Goldsmith is running for the board with Barry H. Stern, head of a Washington real estate firm, and David C. Daneker, a Baltimore attorney.

The incumbents are C. Gay Harrell Jr., executive vice president of Columbia First; James M. Jacobsen, owner of Jacobsen Properties, and Flaxie M. Pinkett, head of a Washington real estate company.