A law firm that has helped represent the D.C. Board of Education in a three-year-old lawsuit against asbestos manufacturers and suppliers abruptly pulled out of its contract yesterday, saying board members were willing to criticize the contract publicly but not to supervise it.

Willie L. Leftwich, whose firm of Leftwich, Moore & Douglas has billed the school system for $4.5 million in fees, said in a letter delivered to the board yesterday that "the confidence necessary for an attorney-client relationship no longer exists." Leftwich also asked a D.C. Superior Court judge to allow the firm to withdraw as co-counsel with the city's own lawyers in the suit.

Leftwich's decision came about a week before the board was expected to cancel the agreement after criticism from D.C. Auditor Otis H. Troupe and board members.

A majority of board members said Thursday they could no longer support the no-bid contract because of its cost, questions about the method of award, the source of its funding and disclosures of ties between the law firm and Delicious Fruit Exchange, a South African cooperative.

The company hired a partner with Leftwich's firm last year to lobby in opposition to trade sanctions against South Africa. Leftwich said the contract ended in October.

Leftwich said in a letter yesterday to school board President Linda W. Cropp (Ward 4) that he had offered to withdraw from the asbestos contract in October and agreed to continue only on condition that the board would do its part.

"My effectiveness . . . has been hampered by the board's repeated refusals to provide me timely direction and supervision," the letter said.

George H. Margolies, legal counsel for the school system, said yesterday the immediate impact of Leftwich's action on the lawsuit "is still to be measured."

The D.C. corporation counsel's office filed the suit in December 1984, seeking to recover damages for asbestos in 2,000 D.C. government buildings, including about 200 schools. A separate claim against a single company is pending.

The city's lawyers serve as the lead counsel and are responsible for proving the source and existence of asbestos in the bulk of the buildings. The school system hired Leftwich's firm to provide legal advice and to help prepare the evidence on asbestos in the schools, a task that some board members had believed would cost $20,000 to $65,000 annually.

Board members have said that Leftwich's firm has nearly completed the documentation but that the city's lawyers are behind in their work.

Board members were at a retreat yesterday and could not be reached for comment. The lawyer handling the suit for the corporation counsel said he could not discuss the case.