"I can remember when I was young," says George Walker, "my mother would not let me come to hear the National Symphony." That was when the NSO played in Constitution Hall, seating for its concerts was segregated, and there seemed to be little incentive in Walker's native Washington for black people to listen to classical music, let alone perform or compose it.

Today, at 57, Walker is the chairman of the music department at Rutgers University, a widely acclained classical pianist (though inactive the last few years) and one of the leading composers in the United States. He was in Washington yesterday for the beginning of the Kennedy Center's week long National Black Music Colloquium and Competition, during which his music will be heard more often than that of any other composer.

Seating arrangements for the National Symphony were only a brief foretaste of the obstacles faced by Walker as he tried to carve out a career for himself in classical music. He began studying the piano when he was 5, and originally planned to be a concert pianist.

"As long as I was a student, everything was all right," he recalled yesterday in a backstage conversation at the Terrace Theatre where the Colloquium is being held.

"I did well in my studies at oberlin, which was known to have a general openness to black people, and after my graduation in 1941, Rudolf Serkin accepted me as a student at the Curtis Institute. The problems developed when I began trying to make a living as a performer." Although he had played concertos with the Philadelphia Orchestra and the Baltimore Symphony and had been enthusiastically reviewed, it was years before Walker was able to get a contract with a major [WORD ILLEGIBLE] management organization. [WORDS ILLEGIBLE] he says, the [WORDS ILLEGIBLE] for me."

"One thing they [the booking agencies] tried was package deals," Walker recalls. "You know the system. In order to get a big-name performer, the sponsoring organization also has to accept a good but unknown one for another program. You want a big name? You also take Walker. Usually, they are glad to accept that kind of a deal, because they want the big name and know the unknown will be good even if he hasn't made a name yet.

"But [the clients] drew the line at accepting me. I guess I was ahead of my time. In 1945, I was getting the kind of reviews Andre Watts got years later, but I had no bookings, no patrons, no help. I struggled for a while and it became evident that if I wanted to survive I would have to go into teaching."

Walker's family was affluent enough to protect him from financial worries and dedicated to promoting his career. "My father supported me until he died," he recalls now. "By then, I was already 32, and I went into teaching for the first time."

Walker is a gentle man, soft-spoken and surprisingly shy for a performing artist and teacher. But a trace of anger creeps into his voice when he talks about the "blatent lies" and discrimination he feels he has encountered in major national competitions and in the early years of his academic life.

"The discovery that ability was not enough to win some competitions hit me very hard," he says. "It took me a long time to recover."

What about academic life? "Not easy," is Walker's answer. "Getting a job at a black college was no problem, but I felt that I should be able to apply to any college." In one case, when he felt that he had been passed over for a less qualified applicant, Walker was angry enough to go to court about it.

In 1960, he got a position at the New School in New York (like Oberlin, noted for its openness), and a year later he joined the faculty of Smith College, where he stayed for seven years after encountering and passing the next barrier -- tenure.

"The real question in a tenure vote," he reflects, "is whether a group of tenured faculty want to live with a person the rest of their lives. At Smith, the music department voted against giving me tenure, then the college tenure committee investigated, overruled the department and supported me."

In the last 10 years, Walker has turned his attention to yet another struggle -- the composer's battle to have his music heard. In this effort he is not sure whether being black is more of a disadvantage than being a contemporary American; but he is sure that both increase the difficulty.

As a pianist who composed piano music, he was able at least to hear some of his work immediately. But he has also composed string quartets, concertos, sonatas for violin and piano, for cello and piano, a trio, orchestral music and choral music. In some cases he has done well. His piano concerto, a Bicentennial commission, received nine performances by four major orchestras while the Bicentennial fever was raging, but he does not know of any performances since 1977 and he has seen no reviews of the Columbia recording. He can't help wishing that the Bicentennial would come around again.

In spite of the discouragement encountered by young George Walker a generation ago, a remarkable new crop of young black pianists has come along; and, since Andre Watts made his breakthrough some of them are making headway. aSeven of them -- winners of regional competitions -- are performing during the Kennedy Center colloquium and competing for a national award.

Walker notes gratefully that many are playing his music. He will not conjecture on whether this is because the music is beautifully written and well-suited to the instrument (although it is), but he says simply that "on the whole, pianists seem more adventurous than composers or string quartets."

Walker says that piano and voice are the areas where black artists have made the most progress -- far outnumbering string players, for example. He believes that the number of pianists reflects a time when "even poor families would own a piano if possible, because the piano was a mark of culture. Sometimes it just sat in the home as a piece of furniture until a child came along who wanted to learn to play it."

As for singers, Walker believes, "they found ready acceptance because, even in the days of slavery in the plantation society, white people thought that black people were good at singing and dancing -- whether or not this is true. In a sense, black singers have been taking advantage of a white stereotype."

Sharing the spotlight with Walker and a wide array of performing artists at the Kennedy Center during the coming week will be 16 other black composers, ranging alphabetically from T.F. Anderson to Olly Wilson and stylistically through the whole spectrum of contemporary idioms in classical music.

Besides the competition for pianists and another for string players, discussions and master classes, the symposium will include several concerts each day.