Politicians who tried to court an increasingly black electorate in Prince George's County with appeals based on racial solidarity were decisively rejected in Tuesday's Democratic primary.

County Executive Parris N. Glendening and Rep. Steny H. Hoyer trounced their primary opponents, 16-year council member Floyd E. Wilson Jr. and Nation of Islam national spokesman Abdul Alim Muhammad, two candidates who directed their campaigns at black voters.

Hoyer and Glendening soundly carried all the black precincts in which Jesse L. Jackson received overwhelming majorities in the 1984 presidential primary. The Jackson campaign included a massive voter registration drive and prompted black leaders to contemplate the election of a black county executive this year.

"The revolution never materialized," said Alvin Thornton, a Howard University political science professor and county resident.

Blacks are expected to represent more than half the population in new census figures, yet increased their numbers in elected office by only two seats, from 14 to 16 out of 54 positions.

Even in the contest where a black challenger, Del. Gloria Gary Lawlah, unseated an incumbent white state senator, Frank J. Komenda, the abortion issue may have affected the outcome as much as demographics in their majority-black south county district, Thornton suggested.

"You can't just jump out and think people are going to vote for you" because of race, Lawlah said. "I think the voters of Prince George's are interested in issues and what you're going to do for them once you are elected."

Muhammad and Wilson, to a lesser degree, roiled the party establishment by focusing on racial pride and political oppression, while incumbents were trying to build coalitions, joining blacks and whites. Some successful black politicians, such as state Sen. Albert R. Wynn and State's Attorney Alex Williams, said those candidates may have misjudged voters in a county with one of the nation's most affluent and best-educated black populations.

Hoyer got 80 percent of the vote to Muhammad's 20 percent, and Glendening got 75 percent of the vote to Wilson's 16.5. Those percentages were reflected in many black neighborhoods. For example, Hoyer received 77 percent of the vote and Glendening got 81 percent at one precinct in District Heights, an area where Wilson and Muhammad had hoped to do well. In another precinct in the inner Beltway community of Capitol Heights, Hoyer got 65 percent and Glendening received 67 percent.

"I think it is dangerous for . . . politicians to jump out here and not be sensitive to voters who are not black," said Williams, who had no primary opposition but faces Arthur A. "Bud" Marshall Jr., the man he replaced, in the general election. "You have to be sensitive and resolve issues. You can't go too far out on a limb. You can't be too black and you can't be too white. You have to be right there in the middle."

Except for Komenda, voters endorsed incumbents as diverse as Sen. Decatur W. Trotter, who won a close race against former state senator Tommie Broadwater Jr., and council member Anthony J. Cicoria, whose record of constituent service helped him win a four-way race despite pending criminal charges of tax fraud and theft.

Cicoria, who said he has retained Mayor Marion Barry's attorney, R. Kenneth Mundy, for his Oct. 1 trial, said incumbents who serve their constituents generally get returned to office, regardless of race or other factors. "Every four years you go in for a contract," he said in an interview yesterday. "If you do a good job, your contract is renewed."

This view was echoed by winning candidates and their backers.

County Council member Jo Ann T. Bell won a majority of votes in a black majority district where she was challenged by businessman and Rainbow Coalition activist Bennie Thayer and by educator Linwood Jones.

"The message is you can't be just a black candidate," said Lance Billingsley, Glendening's campaign manager and head of the county economic development corporation. "You have to be a black candidate who is qualified and has the right position on issues and a record of serving the constituency well."

But Thornton and others said racial politics are almost inevitable in a county experiencing rapid demographic change. "Race is the background music in Prince George's," said Del. Timothy F. Maloney (D-Prince George's).

Some see positive effects of heightened racial awareness. "It means that politicians, black or white, must be cognizant of the issues of the community," said political scientist Thornton. "White politicians are saying, 'Black people, you are significant and I need your vote.' Black politicians, on the other hand, are gaining confidence to step out from under the paternalistic party structure."

Some voters, while acknowledging that independents made weak showings at the polls, said their challenges were significant nonetheless. "The vote in itself may not have been successful. But it does send a message to people that they have to get their acts together," said Maurice House, a Camp Springs school activist.

Politically active lawyer Wayne Curry suggested that the sum of total black votes added up to more than its parts, and that slate candidates are hard to beat.

"It's just very difficult beating a unified slate of incumbents," Curry said. "But the clear message is" some challengers did well and the total of all their votes was impressive, he said.

The biggest challenge for black leaders looking toward 1994, Curry and other observers said, is building a unified coalition among a diverse black constituency.