The National Association for the Advancement of Colored People formally filed a lawsuit yesterday to force the NAACP Legal Defense and Educational Fund Inc. to stop using the NAACP initials in its name, but officials of the association praised the defense fund even as they announced the suit.

NAACP officers also acknowledged that they had already spent a large sum on the suit, including a retainer to Samuel R. Pierce Jr., now secretary of the Housing and Urban Development Department, who was called in to handle the matter in 1979-80 after the NAACP's own counsel advised strongly against pursuing a suit.

Pierce apparently moved to his HUD post before he could make much progress in preparing a lawsuit. The NAACP then passed the case to former Massachusetts senator Edward W. Brooke, who filed the suit in U.S. District Court here yesterday. Brooke, too, has received a large retainer from the NAACP, reportedly $25,000.

Benjamin L. Hooks, executive director of the NAACP, said yesterday that "not one dime" of money contributed to the association by public donors would be used to prosecute the case against the legal defense fund. He said the money would come from chapters around the country.

At a news conference called to announce filing of the suit, Hooks said the legal defense fund was "obviously . . . doing a good job," and refused to criticize it directly.

Margaret Bush Wilson, chairman of the NAACP's national board, said local branches working with the defense fund on specific legal cases "should continue to do so," and she expressed hope that the two groups will continue to cooperate.

However, Hooks did describe the board of the legal defense fund as "a group of outstandingly rich black and white lawyers sitting in Wall Street offices" who did not deserve to speak for NAACP traditions.

The defense fund was established as a separate organization in 1939 so the NAACP could accept tax-deductible contributions to pursue legal cases and educational activities. For 18 years, the two organizations were run by interlocking boards. In 1957 the Internal Revenue Service ruled that they had to be separate.

According to sources in both organizations, they worked harmoniously most of the time, with some differences of opinion. Some NAACP officials were displeased when Thurgood Marshall, accepting a seat on the 2nd U.S. Court of Appeals in 1961, designated Jack Greenberg to succeed him as director of the legal defense fund.

Later, NAACP sources said, there were complaints that Greenberg was too independent and took too much credit for the defense fund while giving the NAACP less than its due for legal victories.

As long as Roy Wilkins was executive director of the association, these sources said, tensions between the two groups were in check. But new NAACP leadership reopened the matter in the late 1970s.

At the time, Nathaniel Jones was counsel to the NAACP in New York. He is now a federal appeals court judge in Cincinnati. Jones told the new leaders of the NAACP that a lawsuit on the issue of the initials would be ill-advised and would face formidable legal obstacles, informed sources said yesterday.

The association's directors then turned to Pierce, a private New York lawyer at the time. According to some sources, Pierce did very little work on the lawsuit idea before joining the administration. He could not be reached for comment yesterday.

Hooks and Wilson said yesterday that the suit is necessary to end public confusion about the two organizations, which sometimes leads to fund-raising problems for the NAACP, and to reclaim the organization's name for the exclusive use of the association.

Vernon Jordan, former executive director of the national Urban League who is now practicing law here, will represent the fund free of charge in the suit with two other lawyers.