A Montgomery County Circuit Court judge yesterday blocked the county police department from starting a class for new recruits until he hears the claims of disappointed white male candidates who say they were unfairly rejected by a lottery.

The temporary injunction handed down by Judge Joseph M. Mathias rerailed, at least for the time being, the county's attempt to hire more women and minorities than ever for the police department. The department had only white male officers until 1968, and its ranks are only 4 percent black and 4 percent women today.

The rookie class had been scheduled to begin next week with 15 white males, eight white women, one black woman, five black males, and one Asian-American male.

A $40 million civil suit filed against the county Thursday, by 15 unsuccessful candidates charged that a lottery used to winnow down an overstocked pool of qualified male applicants was "totally irrational" and "without any correlation to merit or the capabilities of the applicants."

County Executive Charles Gilchrist said the ruling was a setback for the county's affirmative action goals. "I disagree with the implication that we can't make special efforts to attract minorities and women," Gilchrist said. "We can review and refine the methods. The lottery procedure isn't crucial, but he (the judge) says we can't take affirmative action into account."

The judge, however, said he was not ruling on the merits of the disappointed candidates' arguments but merely maintaining the status quo pending further arguments on both sides.

Lawyers for the unsuccessful applicants who have banded together in a group called COPPS (Candidates Opposed to Present Policies and Standards) denied theirs is a reverse discrimination suit.

"We're not saying you can't have affirmative action," said John Walsh, speakings for COPPS. "You would have to do it properly, though."

He added the lottery process violated the county's own requirement that selection be based on merit.

This would have been the second rookie class the county has assembled under affirmative action guidelines adopted informally after black officers in the police department filed a complaint in 1977 with the Equal Employment Opportunity Commission.

Although the department is shorthanded, down nearly 100 people from two years ago and losing 3 percent of the force a month in attrition, the class last spring was smaller than desired because not enough minority applicants were recruited.

The department's Black Coalition president, Michael Jones, said "If there is a sincere effort (and) not a charade, people will come forth."

Before the most recent selection, a four-week promotion campaign was held with advertisements in black newspapers and recruiting talks on college campuses. Nonetheless, after the candidates had gone through preliminary screening, taken a written test and been scored on standardized interviews, white males outnumbered black males three to one.

Of the minority and women candidates, 15 were ranked outstanding, the highest of the county's four categories. They were offered places in the rookie class, which was filled out with outstanding white males chosen by lot. That procedure had been agreed upon by the county personnel department, police department and county attorney.

It was the first time the county had attempted a lottery.

Candiates who received numerical scores ot 90 or above were classified as outstanding, and their numerical scores were dropped.

The county maintains that the lottery in which the names of outstanding white male candidates were picked from a box is the fairest way to choose a limited number of candidates from a pool. County officials say the lottery removes the element of subjectivity from the selection and further refinement of candidates termed outstanding would only degenerate into hair splitting.

Of the 276 white males who showed up for the first screening, 74 finished in the outstanding category. Of 102 black males, five were outstanding. Of 54 white females, eight were found to be outstanding, and of 38 black females, one finished in the outstanding category. One of six Asian-American males was ranked outstanding.

"We could determine no distinct difference between those who were outstanding," said a spokesman in the county personnel office. "We were saying, "We don't care which of the 74 white males (was classified as outstanding.) We'll take our chances with any 15." The differences between them were meaningless."

The county then selected all the women and minorities in the outstanding category and filled the rest of the vacancies in the 30-member class with white males who had been classified as outstanding.

The lottery was held to pick only enough to fill the class.