LIKE MOST cultural institutions, New York's Museum of Modern Art (MOMA) must pay for its still-increasing popularity with increasing financial difficulties. Like many institutions, it would defray the cost of past expansion with more expansion. MOMA's plan for meeting a million-dollar annual deficit is based on the questionable assumption that doubling the size of its present building on Manhattan's West 53rd Street and filling it with twice as many paintings, sculptures, souvenirs and activities would make the museum not only twice as attractive but also solvent. To pay for the expansion, MOMA would demolish some of its neighbors (including a nice Beaux Arts townhouse and the popular Museum of Contemporary Crafts) and build a 52-story, $25 million luxury apartment tower on top of its museum building. Thanks to the clout of its trustees, the scheme was made possible by special New York State legislation (which includes granting the private museum the state power of condemnation) followed by swift approval on the part sundry New York City agencies. The only thing that is holding back the wrecking ball is a pending law suit by the nearby Dorset Hotel, which charges that the MOMA tower would illegally deprive it of light and business.

The law suit, we hope, will give MOMA's trustees time to think. Many New Yorkers are deeply disturbed by this bizarre enterprise. So are we.

It is disturbing, to begin with, that one of the country's leading cultural institutions would do this to itself. Its facade, a rare example of Early Modern architecture (designed in 1939 by Philip Goodwin and Edward Durell Stone, with a later addition by Philip Johnson), is to be obliterated. Worse, Philip Johnson's sculpture garden would be reduced and placed in the shadow of the tower - designed by the currently most fashionable architect, Cesar Pelli. What's more, the apartment-museum complex would destroy the pleasant character of MOMA's present setting - a street of varied five-story buildings that provides welcome relief from the shaded glass canyons along Fifth and Sixth Avenues. To keep Manhattan livable, its planners ruled long ago that there should be no mid-block highrises. Some New Yorkers doubt, furthermore, that MOMA actually would make money on the deal.

But what is most disturbing about this venture is that it may tempt other museums into the real-estate business to supplement their endowment. There are already rumors that the Frick Gallery is thinking of sprouting a money-making tower in its garden. How about converting the upper floor of Washington's Corcoran Museum into a condominium? Or remodeling the Smithsonian into a luxury hotel? Either would make as much sense as what MOMA has in mind.