Duke Zeibert's restaurant and the Brooks Brothers men's clothing store will move out of the La Salle Building at Connecticut Avenue and L Street Nw within six months -- paving the way for construction of a major retail and office complex at the key downtown location.

Retail and business sources said yesterday that developers Theodore Lerner and Albert Abramson have agreed to pay more than $3 million to effectively buy out leases in the La Salle Building of the two remaining major tenatns. Business sources estimated overall costs of the new complex at $100 million or more.

Brooks Brothers, a division of Washington-based Garfinckel, Brooks Brothers, Miller & Rhoads, will move its store to much larger facilities at 1828 L St. NW, about Feb. 1. The store will remain there and not move back to the new complex when it is finished in the early 1980s.

Developers of the Connecticut and L corner have agreed to pay a sunstantial part of the costs of relocating Brooks, including remodeling of the 1828 L St. location into a "first-class Brooks, location, with woodwork and all," said Harold Trimmer Jr., secretary of the Garfinckel corporation.

As 1828 L St., Brooks Brothers will have about 25,000 square feet -- more than double the current La Salle Building location, which ranks No. 1 among all brooks Brothers stores in terms of sales per square foot. In addition, Brooks will occupy the ground floor location in a part of town that has become a major office center for professionals and trade associations.

The future location for Duke Zeibert's 420-seat restaurant -- a popular establishment for lawyers, politicians, athletes and reporters -- was not clear last night. Zeibert has been studying several potential locations to replace his 30-year-old restaurant, now at a location he has called the "best corner in the city of Washington."

"In an interview last night, Zeibert said he is studying one location on 19th Street between L and M Streets NW, being developed by Abramson, but that "it looks awful tough . . . I don't know what will happen." He expressed regrets that he had to move but added that he couldn't turn down the lease purchase deal.

Zeibert said he will close his doors in February or March and hopes to reopen shortly thereafter, elsewhere downtown. But if he can't afford a Washington site, Zeibert said, there may be no more Duke's here.He said offers remain on the table to open a restaurant in Las Vegas and "sometimes people make you offers you can't refuse."

He ruled out a basement location for the restaurant ("I have phobias about going downstairs") or a "hayloft." Zeibert said his major problem has been finding a place big enough to handle his size of operation -- requiring at least 400 seats.

Developers Lerner and Abramson Acquired the La Salle property in mid-1977 in a multi-million dollar transaction. They had hoped to begin tearing down the 50-year old structure earlier this year but were forced to postpone the wrecking balls because Garfinckel and Zeibert insisted on agreement to buy out their leases.

Lerner said yesterday that "as soon as they (Brooks and Zeibert) get out, the building goes down." Lerner declined to discuss plans for the giant complex to be built at Connecticut and L, however.

Partnership details revealed earlier that Lerner and Abramson plan a shopping center as well as commercial offices and a luxury hotel at the site of the La Salle, as well as separate properties around the building -- some 50,000 square feet together.

Lerner and Abramson paid $2.2 million in 1977 to buy 13,000 square feet in four small tracts adjacent to the La Salle, which occupies the southwest corner of Connecticut and L, atop a subway station.