WHEN YOU ask him about the National Endowment for the Arts, Ezra Laderman comes up with a key word that you don't hear often in Washington these days: sanity.

"Crazily enough," he says, "at the endowment I met some of the sanest people I have ever met in my life."

Laderman left the NEA a few months ago, after serving as director of its music division for three years and as a member or chairman of various music panels for nearly a decade. It was a quiet departure--not at all the Alexander Haig kind of thing. It was so quiet that for months you would not know he was gone unless you called up and asked for him.

But he insists that there was no friction; he left "with lots of hugs and kisses," and the sweeping changes that he introduced are proceeding on their planned course in the care of people who helped him to set up the new policies. "A lot of me is still there," he says wistfully. He is keeping his apartment in Columbia Plaza, a few minutes' stroll from the NEA office, where he used to compose in the morning before going off to his bureaucratic duties.

"I left because I must compose," says Laderman, who originally accepted the endowment post from former NEA chairman Livingston Biddle with the understanding that he would remain active as a composer. That priority was respected throughout his tenure, although scruples about conflict of interest held down the number of commissions he would accept.

He knew his days at the endowment were numbered when bureaucratic problems began coming to bed with him.

In the beginning, he recalls, "all the years I composed, when I put my head on the pillow at night, before falling asleep, I would be organizing in my mind what I would be tackling creatively tomorrow morning. I didn't realize that this was what I was doing, but I'd be turning over in my mind where I'm at in the work that I'm composing. It was like playing a game of chess and knowing where all your options are. I was on top of the situation."

Then, during his last year at the NEA, "through the trauma of defending the very life of the endowment, I found that when I went to sleep at night, what I would be thinking of had only to do with the endowment . . . This had never happened before, and then I realized that I must think about separating myself from the endowment."

The connection began in the early '70s, when Laderman served the endowment in a volunteer capacity, first as chairman of the composer-librettist program and later as a member of the music advisory panel. In his three years at the top, as director of the music division, Laderman energetically began programs in many areas--chamber music, solo recitalists, new music, and the recording of American music, among others.

Even more important, perhaps, he changed the rules of the game. When he began, grant awards were made more or less routinely on the basis of budget size. Under the new system, applicants are examined by panels of their artistic peers, with a rotation system to prevent the establishment of cliques and favoritism.

As Laderman explains it, "excellence" became "the bottom line. Every institution was evaluated, both musically and administratively . . . There was a much greater burden, making proper evaluations. But it gave credibility to what the endowment was doing, and we were able to lift the level of excellence and stand for something we felt was important." During these years, the budget also escalated enormously. The procedural changes are probably permanent; the budget is in serious trouble.

Meanwhile, as a composer, Laderman's business is booming. In January, the Philadelphia Orchestra will give the world premiere of his Concerto for Flute and Bassoon, and in March the National Symphony will premiere his Fifth Symphony, with an important solo part for Galina Vishnevskaya. He is now working on a new composition for the Denver Symphony (commissioned on the day he left the endowment) and he has accepted a commission for a double string quartet to be premiered at the Library of Congress.

Laderman uses a variety of composing styles, but the general flavor of his music is distinctively modern, often dissonant and sometimes atonal or pointillistic. It will be interesting to see how his music is accepted in Philadelphia, where the prevailing tastes in music as in other areas have long been conservative. For now, all he will say on the subject is, "It was a challenge for me to write a work that would be accepted in Philadelphia."

Composing, Laderman says, is "an ongoing affair that I've had ever since I was a little kid. I knew early on in my life . . . I didn't know how I'd ever make a cent, but I knew that I was going to compose. I thought maybe I'd be a piano teacher." He began composing in Brooklyn when he was 6 or 7 years old, and still has a piece that he wrote when he was around 9 or 10, in 1933 or '34: a setting of Wordsworth's poem "Daffodils" for voice and piano.

"I grew up with a whole group of young people who were also determined to make careers in music," he recalls, "through high school, through the war and into our twenties in New York. In high school, I wrote a piano concerto. There was a whole gang of 15- and 16-year-olds who knew of me as a composer." Today, these musicians are all over the country, among the most renowned musicians of our generation.

In the army, after helping to liberate Leipzig, he wrote a "Leipzig" symphony that was premiered by the GI Symphony in Paris (it was his first experience with federal support for the arts) and performed a number of times elsewhere in Europe. He finished the war as orchestrator (something like a composer in residence) with the GI Symphony, and found himself working as a professional musician almost by accident.

Now that he is back to composing full time, offers are pouring in for new jobs in music administration, and he is seriously considering several that will allow him to continue composing--preferably in the Washington-Baltimore area so that he can stay in his apartment near the NEA. Now in charge of the endowment's music programs is Adrian Gnam, a conductor and oboist (formerly first oboe of the Cleveland Orchestra under George Szell), whom Laderman hand-picked as his assistant and later as his successor.

Laderman's parting shot at the NEA turned out to be a love pat for the present chairman, Frank Hodsoll, a figure on whom most people in the arts seem to be reserving judgment. Laderman's impressions after working closely with Hodsoll may be encouraging to others.

"He is bright," says Laderman about his ex-boss, "one of the quickest learners I have met. He is affable; he listens; he doesn't shoot from the hip. He's slowly making up his mind philosophically as to the direction he wants to take the endowment. I think he's a workaholic. I think if anything, so far, he shows a bias toward what is innovative, what is truly creative, what is new, what is a departure from the conventional, and this we had no way of knowing before he came."