The U.S. Judicial Conference yesterday recommended that federal judges not belong to organizations that discriminate in their memberships.

The action by the conference, which makes policy for the federal judiciary and is headed by Chief Justice Warren E. Burger, follows several years of controversy over the membership of judges in all-male or all-white clubs.

The conference said it was "inappropriate for a judge to hold membership in any organization that practices invidious discrimination." But as is its usual practice, the conference left it to the "conscience of the individual judge whether membership in a particular organization is incompatible with the duties of judicial office."

Burger, who announced the action yesterday, is listed as a member by the all-male Alfalfa Club in Washington, as are Associate Justices Potter Stewart, Lewis Powell and William H. Rehnquist. A spokesman for Burger said that any club memberships were "his personal business."

The conference, which includes, in addition to Burger, 25 top federal judges from around the country, said in its amendment that "a judge should carefully consider whether the judge's membership . . . might reasonably raise a question of the judge's impartiality in a case involving issues as to discriminatory treatment of persons on the basis of race, sex, religion or national origin."

The conference has no way of enforcing its code of conduct. But Ann Macrory, a civil rights attorney who has led an effort to get nominees for judgeships to resign from discriminatory clubs, said the amendment would be useful in disqualifying from hearing discrimination cases any judge who remains a member of such a club.

Many high public officials, including Attorney General William French Smith, belong to all-male clubs, and Smith's membership was a point of controversy in his Senate confirmation hearings. Controversy also erupted last year in the Senate Judiciary Committee over judicial nominees who belong to all-male and all-white clubs. Some nominees resigned from such organizations under pressure.

The conference left a loophole for judges who choose to use it, however. The amendment said that membership in clubs practicing "invidious" discrimination was inappropriate and that "the question of whether a particular organization practices invidious discrimination is often complex and not capable of being determined from a mere examination of its membership roll."