President Carter today urged black Americans to participate more actively in the political process, declaring that the hold of the "special interests" on Congress will be broken only by an end to voter apathy.

Delivering the commencement address at Cheyney State College here, the president said that only through the political participation of "ordinary citizens" will Congress be prodded into passing legislation significantly changing American society.

"Our problems are serious and they are real," Carter said. "They will not disappear if we ignore them. There are powerful special interests in our country which feed on the apathy of ordinary citizens.

"They enjoy the special privileges of power and influence, and they change. Those among you who believe in change, in improvement - those who are committed to making our country greater - must let your voices be heard if we are to defeat these special interest."

The president's attack on the "special interests" was a familiar theme, one he was used from the earliest days of his administration. But in his speech today to the 200-member graduating class of the predominantly black college, Carter's real target appeared to be Congress, which has continued to frustrate a number of White House proposals.

Acknowledging his own debt to black voters - "You and other black Americans are the reason that I am president," - Carter gently scolded those in his audience who acknowledged not voting in last fall's congressional elections.

"Both political candidates and incumbents have got to know that you will vote. How are we going to have the leadership to fight for equal opportunity and affirmative action in jobs, schools and housing if even the act of voting is too great an effort?"

He said that while the 25-year-old Brown v. Board of Education Supreme Court decision outlawed school segregation, racial barriers in the schools did not really begin to crumble until after enactment of the Voting Rights Act of 1965.

It will take the same kind of pressure, through the ballot box, to gain passage of legislation to expand federal authority to halt housing discrimination, Carter said.

"We are not going to get this change through Congress without your political participation," he said.

"The very least we need is your vote for members of Congress who will enact laws that can eliminate discrimination and give us the enforcement powers we need."

Carter prefaced his remarks about voting by taking an informal poll of the audience, first asking how many cared deeply about various issues, and then how many voted last November. He seemed surprised when a large majority in the audience raised their hands to indicate they had voted and told them it was "much better than the national average."

Voter turnout, particularly among blacks and other traditionally Democratic groups, is likely to be crucial to the president's chances of reelection next year.

The audience of several thousand responded politely to the speech, delivered in the main quadrangle of the college on an overcast and damp morning. The president flew from the presidential retreat at Camp David, Md., although the poor visibility forced his helicopter to land in Philadelphia, about 20 miles northeast of here, rather than on the college football field as planned.

Arriving at the college by motorcade, Carter passed about 50 protesters calling for an end to nuclear power.

Today's speech was the third commencement address Carter has given as president. The other two, at the University of Notre Dame and the U.S. Naval Academy, dealt with foreign policy.

Unlike his earlier commencement addresses, however, the president today did not attempt to make a basic policy pronouncement, but concentrated instead on an appeal for allies in his frequent struggles with Congress.

"Our country does face extremely difficult problems-problems like inflation, energy shortages, inequality, discrimination, unemployment and worldwide threats to peace. Each of these problems brings with it a tendency toward withdrawal from responsibility and crippling fears - fears that stand in the doorway between us and needed solutions."

But, Carter said, these problems will not be overcome if they are ignored by the majority of Americans.

The president made no announcements during the speech, but did promise to review an earlier administration decision to cut $200 million from the 1980 summer jobs for youth program. CAPTION: Picture, President and Mrs. Carter at commencement exercise at Cheyney State College in suburban Philadelphia.