In the second go-round of their hunt for a president, the trustee of the University of the District of Columbia have narrowed their search to three men, sources said yesterday.

Although the trustees have not yet offered the job to any of the three, the sources said the man with the most support among the 16 board members is Lisle C. Carter Jr., 51, a former assistant secretary of Health, Education, and Welfare who now is chancellor of the Atlanta University Center.

The other two being seriously considered for the post, sources said, are Prezell R. Robinson, 55, president of St. Angustine's College, Raleigh, N.C., and Herman R. Branson, 62, president of Lincoln University near Philadelphia.

The trustees said they expect to name a president by Monday, the day when the city's three public colleges - Federal City College, Washington Technical Institute, and D.C. Teachers College - are scheduled to merge to form the new University of the District of Columbia.

This is the second time the trustees have tried to pick a university president. In mid-June, the man who was their first choice Randolph W. Bromery, chancellor of the University of Massachusets at Amherst, turned down the job.

Yesterday the sources said the trustees would not make a definite decision on who they want to be president until after they are sure that the person they choose will take the job.

Carter, who is in Washington this week for the national convention of the Urban League, said he met with the trustees on Sunday, but he said it would be "presumptuous" of him to say whether he would take the president's job.

"I have no problem with them considering me," he remarked in an interview, "but I am not a candidate for the job."

Since 1974, Carter has been head of the university center in Atlanta, which manages joint programs, facilities, and fund raising for six predominantly black colleges with a combined enrollment of about 7,5000.

A lawyer who was executive director of the Washington Urban League in the 1950s, Carter served as assistant secretary of HEW in the Johnson administration from 1966 to 1968 and later was a professor of public policy at Cornell University in New York.

Robinson, who has Ph. D. in socilogoy from Cornell, has been president of St. Augustine's College for 10 years. The college, which is affiliated with the Episcopal Church, has about 1,600 students, nearly all of them black.

Branson has served as president of Lincoln University, another predominantly black school wwith about 1,2000 students, sicne 1970. He taught physics at Howard University here from 1941 to 1968.

In June, Branson was one of four finalists recommended by a search committee for the UDC post before the trustees chose Bromery. Yesterday, he said he was "flattered" that he was being conordered again.

Sources said the trustees are not reconsidering Cleveland L. Dennard, the former president of the Washington Technical Institute even though they have received a substantial number of letters supporting him.