Three prominent black leaders described Robert H. Bork yesterday as "too risky" to serve on the Supreme Court as the judge suffered the first defection among Senate Republicans in the fight over his nomination to replace retired Justice Lewis F. Powell Jr.

Sen. Bob Packwood (R-Ore.) announced he would fight the nomination.

The black leaders -- William T. Coleman Jr., transportation secretary in the Ford administration; former representative Barbara Jordan (D-Tex.), and Atlanta Mayor Andrew Young -- said Bork has consistently opposed the expansion of rights for minorities that were won during the civil rights struggles of the 1950s and 1960s.

"Here you have a man who in every instance on these great issues, publicly as a scholar, always comes out the wrong way," Coleman told the Senate Judiciary Committee as it resumed hearings on Bork's nomination.

The committee heard testimony about a divided American Bar Association's judicial evaluation committee, which found Bork "well qualified" for the high court. However, four of the 15 members found Bork "not qualified," the first time in 15 years a Supreme Court nominee has not received unanimous approval of the lawyers' group.

Sen. Orrin G. Hatch (R-Utah) harshly attacked the dissenters, saying they opposed the nomination for "basically political reasons," adding that several of the dissenters were liberals or affiliated with civil rights groups.

Harold R. Tyler Jr., the head of the ABA evaluation committee, a former federal judge and deputy attorney general in the Ford administration, defended the integrity of the four committee members who voted Bork "not qualified." Tyler testified that "I think they acted in good faith."

The dissenters cited "concerns as to his judicial temperament -- for example, his compassion, open-mindedness, his sensitivity to the rights of women and minority persons or groups and comparatively extreme views respecting constitutional principles or their application," Tyler had said in a letter to the committee.

The reasons cited by the four ABA committee members are among the major themes being used by Bork's opponents in an attempt to defeat the nomination.

As the Judiciary Committee began what is expected to be two weeks of testimony from prominent supporters and opponents of Bork, Packwood became the first GOP defector from Bork's cause, announcing that he will vote against the nomination and, if necessary, take part in a filibuster to defeat it.

Packwood, one of the Senate's strongest supporters of a woman's right to abortion, said that, after listening to Bork's testimony last week and meeting privately with him twice, he had become "absolutely convinced that he would try to overturn" the Supreme Court's 1973 decision legalizing abortion.

"I am convinced that Judge Bork feels so strongly opposed to the right of privacy that he will do everything possible to cut and trim the liberties that the right of privacy protects," Packwood said in a statement.

Bork's opponents saw Packwood's action as a sign that they may be able to pick up a handful of moderate Republican votes considered critical to the confirmation. The most pivotal, Judiciary Committee member Arlen Specter (Pa.), said he is still undecided. Others who have been mentioned include Sen. Lowell P. Weicker Jr. (Conn.), the only Republican in the Senate to vote last year against the elevation of William H. Rehnquist to chief justice, and Sens. John H. Chafee (R.I.), Robert T. Stafford (Vt.), Mark O. Hatfield (Ore.) and John Heinz (Pa.)

Administration officials said the announcement had been anticipated for weeks, given Packwood's statements in July that he would probably oppose the nomination, and predicted it would not influence other GOP votes.

"It's entirely possible we'll hold all the other Republicans," said Assistant Attorney General John R. Bolton.

In an interview, Packwood said he hopes a filibuster will not be necessary but that he will resort to the tactic "if there is a risk that {Bork} would be confirmed."

"I told {White House chief of staff} Howard Baker two weeks ago that I had to be convinced 'beyond a reasonable doubt' " that Bork would not vote to overturn Roe v. Wade and other rulings based on a right to privacy," Packwood said. He said that Bork "qualified" and softened his longstanding positions in some key areas, but "not on privacy."

Bork was supported yesterday by a panel of former attorneys general, including Edward Levi, who served under President Gerald R. Ford, and William French Smith, President Reagan's first attorney general.

Levi said Bork "has an inquiring, powerful mind; he cares about our society and about people." Responding to charges that Bork has altered some of his controversial positions to win Senate votes, Levi said, "One of the consequences of having an inquiring mind is that you do change position."

Smith charged that Bork had been subjected to an unprecedented campaign of "misrepresentations, distortions and lies."

"There are people in positions of responsibility in government who are lying," he said.

Coleman, Jordan and Young said Bork's confirmation would risk reopening issues that were settled years ago.

"My word," Jordan said, ridiculing Bork's assertion that there was "no theoretical basis" for the Supreme Court's landmark series of "one-man, one-vote" rulings. Recalling how she lost her first two political races before the court rulings forced the Texas legislature to reapportion itself, Jordan said that if Bork's view had prevailed, "I would now be running my 11th unsuccessful race for the Texas House of Representatives."

Jordan said she was "incredulous" at some of the more moderate views Bork voiced in his testimony last week and that she would give "little weight" to them.

Young said the economic growth of Atlanta and other cities of the South "could not be imagined in a segregated society or one where the race question was still open."

Calling the economic transformation of the South a legacy of the civil rights movement, Young said, "That is a legacy that is still in progress, still taking shape, still fragile, one we are not willing to run the risk of being tampered with."

Young also argued that last year's congressional elections, when Democrats recaptured control of the Senate, produced a strong popular mandate to resist Reagan nominees such as Bork.

Coleman, a Republican, said, "In this day and age, can we really take the risk of a man who fails to recognize the fundamental rights of privacy and liberty?"

Referring to the Supreme Court rulings "where black Americans won tremendous victories," Coleman said, "On each one, Judge Bork saw fit to criticize the decision."