Eleanor Holmes Norton, head of the U.S. Equal Employment Opportunity Commission, called on graduates of the new University of the District of Columbia yesterday not to become so wrapped up in middle class ambitions that they "walk away" from the plight of the hard-core black poor.

"You who graduate today belong to the privileged black world of the college graduate," said Norton. "But in that other black world there is a new and angry underclass ...a virtual pariah class, a minority among minorties...a class often more despised than pitied."

"Lacking a work tradition and often even a work aspiration, they command less sympathy from both black and white America," Norton said.

"Will you leave them behind?" she asked. "Will the new, black, middle-class college graduate, hungry for the security denied past generations, simply walk away from these last and lost blacks?"

A civil rights and constitutional lawyer who served for seven years as New York City's human rights commissioner, Norton, 40, said her own "sit-in" generation "knew we were part of one people, and privilege for a few could not substitute for freedom for us all." She indicated less confidence in the commitment, and the "attention span," of the current generation of black college graduates.

"I'm a little disturbed," Norton said afterwards, "at this notion that it's conceivable that young people in growing numbers could become very security-oriented precisely because they come from a tradition where people never had a sense of security."

The commencement exercises, held at the D.C. Armory, marked the first time that students from the old Federal city college, D.C. Teachers' College and Washington Technical Institute were graduated together. But the three schools, while merged on paper into a single university, continue to have separate academic programs and had differing graduation requirements this year. The full merger will go into effect next year, according to UDC officials.

Yesterday's ceremonies, attended by about 750 of UDC's 1,200 graduates and about 5,000 of their relatives and friends, came three days after Congress decided to postpone the start of work on UDC's proposed $56 million Mount Vernon Square campus.

That decision was part of a compromise reached between congressional conferees and D.C. officials over the long-stalled D.C. convention center, also to be located near Mount Vernon Square. Work on the UDC campus was held up until the city produces a new master plan for the university and revised estimates of furture enrollment.

Lile C. Carter, Jr., president of UDC, referred to some of the school's problems as re introduced Norton. "We have been through a difficult year in many respects but also a rewarding year," he said.

In her commencement address. Norton told the overwhelmingly but not exclusively black group of graduates, "You represent all the lost poets and mathematicians, engineers and architects, nurses and business people who never were because they were black."

"The talent of generations lost to us because of the barriers of color is alive in you," she said. "It is a heavy burden that you bear. It is made heavier by the continuing awful contrasts in our community."

American University held simultaneous commencement exercises at Constitution Hall yesterday, with 2,040 graduates. Speakers included George Stevens, Jr., director of the American Film Institute, Bertrand de Jouevenel, economist and philosopher, and journalist I. F. Stone, who received an honorary doctorate in humane lettesr.