Leo M. Bernstein, a wealthy Washington investor and banker, is relinquishing control of Women's National Bank and selling his family's shares to a small group of investors headed by the bank's current president, Barbara Blum, Bernstein said yesterday.

The bank will continue to operate as a minority bank as it has under Bernstein and previous owners, making it eligible for special deposits that have been set aside by federal and state governments and large corporations as part of an effort to increase the viability and soundness of minority banks. To be a minority bank, 50 percent of the bank's ownership and officers must be members of a minority group or be women.

The sale, at an undisclosed price, will mark the first time in nearly 20 years that Bernstein has not controlled or shared control of a District bank. Ever since he helped found D.C. National Bank in 1966, Bernstein has been a key banking figure in Washington, first as president and chairman of D.C. National, then as vice chairman of National Savings & Trust Co. and later as chairman of Washington National Bank, which was merged into Security National Bank, where Bernstein remained as chairman until late last year.

Bernstein purchased 51 percent control of Women's National Bank -- the first nationally chartered women's bank in the country -- in April 1983 for $1.3 million, transferring the bulk of his shares to his wife and granddaughters to permit the bank to maintain its minority status.

At the time Bernstein acquired control, the bank was expecting a substantial loss -- its second in two years. Bernstein said yesterday that he bought the bank after he was approached by its chairman, who asked him "to serve as a consultant in an effort to return the bank to a stable financial position."

"After two years of effective management, I am proud to say the bank has grown from $14 million in assets to its current status of $30 million, and has been consistently in the black in the first half of 1985," Bernstein said. "I am now happy to turn WNB" over to a group headed by Blum, who served as the deputy administrator of the Environmental Protection Agency under President Carter. Blum became the bank's president in September 1983.

However, Bernstein noted, "I shall remain available, for a short time, as a consultant" to the bank.

Blum said that Bernstein, 70, notified her earlier this month that he wanted to sell his shares. "His primary interest now is in antiques, hotels and real estate he owns in Virginia. . . . He wanted out of anything that confines him," she said.

Blum said Bernstein gave her until Aug. 1 to come up with an offer for his shares. In addition to Blum, the other investors include Kerry H. Stowell, founder and presdient of several companies, including So-Sound, a music publishing firm; Richard W. Naing, president of the real estate development firm Wynmark Development Corp., and Mark G. Griffin, senior partner in the Washington law firm of Lambert & Griffin.

Earlier this year, Blum noted that since Bernstein severed his ties with Security National Bank in November and was able to devote full attention to Women's National Bank, the bank has experienced substantial growth. Within six months, she said, assets increased from $23 million to $26 million and deposits grew from $20 million to $23 million.

"Bernstein brought a credibility to this bank because his involvement attracted a lot of people who wouldn't have come here because they thought it was a sissy bank," Blum said earlier this year.

That credibility should not change, Blum said, because Bernstein will remain as a consultant. In addition, his chief assistant, Shirley Thompson, will join the bank as a vice president in charge of business development.