A combative Congressional Black Caucus opened its ninth annual legislative weekend meeting here yesterday with expected attacks on President Carter and unexpected, blunt warnings to potential presidential candidate Sen. Edward M. Kennedy (D-Mass.) not to take the black vote for granted.

To underscore its dissatisfaction with Carter and avowed political aloofness toward Kennedy, the caucus leadership made a point of not inviting either man to speak tonight at its annual dinner, one of the major black political events of the year.

This means that, for the first time since assuming office, President Carter will not be a keynote speaker at the main caucus event.

"We don't want anyone, Carter or Kennedy, to think they own us," said Rep. William L. Clay (D-Mo.), chairman of this year's caucus meeting.

"The decision was made that people who are announced candidates or potential candidates should not use that forum to promote their candidacy," said Rep. Parren Mitchell (D-Md.), former caucus chairman."Otherwise, we'd spend the whole dinner hearing speeches from candidates."

According to caucus and other sources, the decision Mitchell spoke of came during Labor Day week at a meeting in which Clay and several other caucus members threatened to tell the White House that Carter would not be welcome to tonight's dinner, even as a drop-in guest.

The sources said that more moderate caucus members blocked that action.

Asked if the president would attend the dinner, a White House spokesman said last night: "I don't know whether the president will be able to make it or not. You'll have to check with us tomorrow."

Should Carter come, he could run into two of his expected challengers, Kennedy and California Gov. Edmund G. (Jerry) Brown Jr., each of whom has bought a table.

The caucus' political pique stems largely from perceptions among many blacks that Carter has failed them and from black leadership fears that the rank-and-file will flock to Kennedy without closely checking his credentials.

Civil rights leader Jesse Jackson, in a 30-minute speech that drew standing ovations from an overflow audience at the opening caucus meeting, urged blacks to keep both Kennedy and Carter at a safe political distance.

"They didn't make us," Jackson declared. "Carter didn't make us. Kennedy didn't make us. We made them."

There is no fundamental ideological difference between Kennedy and Carter, Jackson said. So, he said, blacks should not give their votes to Kennedy "for a handshake."

"We want a contract," Jackson declared.

Similar comments were heard yesterday in the caucus' 20 legislative workshops and numerous social gatherings that mark this year's weekend meeting.

Also heard were strong expressions of support for former U.N. ambassador Andrew Young, who announced his resignation Aug. 15 in the wake of controversy over his meeting with a representative of the Palestine Liberation Organization.

Tonight the caucus will present its first international peace and justice award to Young in recognition of "an individual who has made the greatest contribution in bringing a black perspective on international affairs to our country and to the world."

As guest of honor, Young will deliver tonight's keynote speech.