It is difficult to believe that New Yorkers could ever get exercised about too many skyscrapers, but that's what's happening.

In a period of unparalleled construction in midtown Manhattan, there is a growing concern that a new generation of skyscrapers will soon block the sun, degrade the already frightening air quality, disrupt the delivery of city services, and increase the population density of the area to an unacceptable level.

A newly formed group, The New York Committee for a Balanced Building Boom, has retained a former New York state commissioner for environmental affairs and his law firm to help challenge current construction statutes and lobby for constraints sure to anger builders.

"We are not trying to stop the building boom, because that's good for the city," explained Peter Berle, the former commissioner who now represents the group. "But there is an overall problem of proliferation of density in a way that could endanger midtown as we know it and no longer make it an exciting place to be.

"What we currently have is a patchwork of special districts . . . and no one yet has taken a comprehensive look at all of it," he continued. "That's what we want to do."

Berle and his clients face long odds. The recent building boom represents a big shot in the arm for New York, one desperately needed here since the bottom fell out of the real estate market in the early '70s and the city dropped to its knees financially. Anything that spells a reversal of New York's fiscal fortunes has been welcomed in most quarters.

Today, there are 21 major buildings either under construction or on the drawing boards in midtown. Most are clustered in the blue-chip corridor running from 40th to 60th streets, and from Fifth Avenue east to Third Avenue.

IBM and AT&T are building within a block of each other, in the mid-50s along Madison Avenue. Piggyback construction projects are slated over the Museum of Modern Art on 53rd Street and the Racket and Tennis Club on Park Avenue.

Harry Helmsley, a developer, is adding a 51-story tower over the historic Villard houses on Madison Avenue at 50th Street to create the Palace Hotel. And across the street, Saks Fifth Avenue is planning to put up a building over its present store.

Most controversial is the 58-story, $100 million retail and condominium structure schedules for the corner of Fifth Avenue and 56th Street, the site of the old Bonwit and Teller building, by developer Donald Trump. To some, Trump's design would shatter the already fragile character of the area, and signal the final onslaught of glass-and-steel skyscrapers along an avenue acclaimed for its gentility and graceful older buildings.

To its supporters, the Trump project and others like it in the midtown area are to be applauded as the providers of thousands of jobs and the source of a new vitality. No one at City Hall, for example, has been heard carping overly about the aesthetic shortcomings of the high-rise projects.

"I believe in paying bills, and if it's not going to come from the midtown area, where is this city going to make its money?" asked Peter Solomon, deputy mayor for economic development.

"These people [the developers] aren't going to build in Queens or Brooklyn," he continued. "If we say to them, 'Please build over there,' they're going to take off for Montclair or Stamford. Do we really want that?"

Harry Helmsley couldn't agree more. "The businessman wants to be near the rest of the business community," he said.

The man whose job it is to deal with the high-rise issue, at least until he moves to City Hall later this year to become one of Mayor Edward Koch's deputies, is Robert F. Wagner Jr., currently the chairman of the city planning commission. To Wagner, the issue is not so much what goes up, but where. He thinks that tower construction should continue in the midtown area, but in the East Side corridor where towers are already the dominant architectural presence.

"The central challenge is not to stop construction, but to [spread] it to less-developed parts of midtown Manhattan," he said recently.

One of the most potent administrative tools available to Wagner and whoever succeeds him at the planning commission in working for a shift in the direction of high-rise development is tax abatement, which is what rekindled interest in the East Side corridor. Also, the F A R (floor area ratio) bonuses the commission doles out to developers in exchange for public amenities like plazas and open-space setbacks can be used, Wagner feels, to attract builder capital west and south of the corridor.

Wagner has commissioned a study of the current boom, its implications for the future of the area, and the options for altering its course. Admirable, says Berle, but he adds that a lot of developers will win approval for new and objectionable projects between now and the time the study is completed, which could be almost a year from now.

Berle, William Hubbard, who is the motivating force behind the Committee for a Balanced Building Boom, and the committee's supporters aren't going to wait. They are about to demand that the planning commission require developers to prepare environmental impact analyses of major projects before ground is broken.

No such analysis was made of the Trump project, Berle said, or of any of the other large buildings slated for midtown. As a former commissioner of environmental affairs, he has an intimate understanding of the state statutes governing such studies, and says he is prepared to persuade the planning commission that they belong in the midtown construction process.

All of the ingredients are present for a real slugfest. Berle and his group, which includes many a sophisticated and well-credentialed New Yorker, will confront the likes of Harry Helmsley and Donald Trump, economic giants in City Hall eyes.