Amid the glitter of the Congressional Black Caucus' legislative weekend here, one question unsettled the elite group of black politicians, activists and civil rights leaders: Are American black leaders out of touch with the majority of blacks?

A recent poll reported that the opinions of 600 average blacks questioned were vastly different from those of 105 black leaders, including heads of the caucus, the NAACP, the National Urban League, Operation PUSH and the National Conference of Black Mayors.

The survey, published in the American Enterprise Institute's Public Opinion magazine, drew an angry response from black politicians.

"We're obviously not out of touch," said Rep. Mickey Leland (D-Tex.), caucus chairman. "We have the safest seats in Congress . . . . People in our constituency may not understand all that we do, but they are sophisticated enough to know what we do and they keep reelecting us . . . they give us a certain latitude."

The Rev. Jesse L. Jackson, who captured more than 80 percent of black votes in last year's Democratic presidential primaries, said, "Number one, consider the source of the poll. AEI is a right-wing institution.

"Second, they took a small sample. And, third, black leaders are always in front of black people. That's what leadership is. Frederick Douglass was in front on ending slavery . . . . A. Philip Randolph was ahead, Adam Clayton Powell was said to be too brash, Martin Luther King Jr. was clearly in front."

NAACP Executive Director Benjamin L. Hooks questioned the poll itself.

" . . . They didn't poll me, John Jacobs of the Urban League , Joe Lowery of the Southern Christian Leadership Conference or Mrs. Coretta Scott King, so I don't know what black leaders they are talking about," he said. "But even so, you would expect some difference between leaders and their followers . . . . "

Few black Republicans attended the caucus dinner.

Sen. Strom Thurmond (R-S.C.) and Armstrong Williams, a congressional liaison for the Equal Employment Opportunity Commission, stopped by before it began to see King.

Republicans, led by President Reagan and Clarence M. Pendleton Jr., chairman of the U.S. Commission on Civil Rights, have sought for several years to make the point illustrated by the poll.

"We all feel vindicated," Pendleton said yesterday from his home in San Diego. " . . . Black people don't want these leaders left over from some other time; they don't want them . . . black people are more sensible than these so-called leaders. They don't want busing; they don't want preferential treatment.

"They live differently . . . 59 percent of the leaders have incomes over $50,000, but only 3 percent of black people make that much.

"That is why the poll shows that 86 percent of black Americans said they don't belong to any civil rights organization," Pendleton said. "Name one young black civil rights leader. There aren't any because they can't fit in with those liberals."

Political analysts joined black politicians in questioning the validity of the poll, which was conducted by the Center for Media and Public Affairs, a public-policy group.

Eddie Williams, head of the Joint Center for Political Studies, said a poll by his group and the Gallup organization last year had sharply different results.

While 77 percent of blacks in the AEI poll opposed preferential treatment for blacks in hiring, 27 percent did so in the JCPS poll. The AEI poll found that 66 percent of blacks say they think that they have made "progress," but the JCPS survey found only 37 percent of blacks with that opinion.

"We think it is highly unlikely that such a dramatic shift occurred among blacks between 1984 and 1985," Williams said.