Civil rights leaders yesterday accused the Reagan administration of adopting a "caveman ethic" in its civil rights and budget policies, and seeking to turn back 20 years of progress toward racial equality.

John Jacob, president of the National Urban League, said that in Reagan's second year, "things look very grim indeed."

Delivering the keynote address to the 32nd annual meeting of the Leadership Conference on Civil Rights, a coalition of 157 civil rights, labor and religious organizations, Jacob said, "By its civil rights record alone, this administration must go down as the most retrogressive in memory. Even Richard Nixon never tried to go this far."

Elsewhere, however, assistant attorney general William Bradford Reynolds, head of the Justice Department's civil rights division, defended the administration's civil rights record, saying that court-ordered school busing and racial quotas have not worked and that it is time to try new techniques.

In a speech to the Delaware Bar Association, he said, "We are not against desegregation. Any student desiring to attend a public school with students of the opposite race should be afforded the opportunity to do so . . . .

"But at the same time, we will not deprive students of the significant benefits of attending school in their own neighborhoods by insisting on a mandatory, race-conscious transportation remedy that has proven ineffective and holds out little promise for an enhanced educational experience," he said.

Reynolds added that in the area of affirmative action, the administration believes in a "color-blind ideal of equal opportunity for all . . . . We will not tolerate preferential selections that favor less qualified employes over those who are better qualified solely on the basis of a person's membership in a particular racial group."

Back at the Leadership Conference, Eleanor Holmes Norton, head of the Equal Employment Opportunity Commission in the Carter administration, sharply attacked Reynolds as "the major negative force" in the Reagan administration. She called the various positions he has taken "an unprincipled attack on the rule of law" and blamed him for administration efforts to give tax exempt status to segregated private schools.

Jacob charged that both the budget plans and the New Federalism policies of the Reagan administration are turning back the clock on civil rights progress. "The new budget is an even more vicious budget than last year's," he said, while the New Federalism, turning over programs to the states, amounts to adopting "a caveman ethic that is a moral insult to America's ideals.

"The president has often said that if people don't like the way their state governments treat them, they are free to move elsewhere . . . The new states' rights practically invites states to export their poverty by making conditions so tough for their poor that they'll want to move elsewhere," he told the meeting.

"We have to label the New Federalism for what it is--a prescription for inflicting further misery on the poor and for splintering national issues. It is the old notion of states' rights. Those programs and powers came to Washington because of state abuses, and turning them back to the states is bound to lead to new abuses," he said.

But despite the problems, Jacob said the civil rights movement "may be in a better position than ever before to enlist not only the white poor but also many white middle class people who feel insecure and threatened by the holes being drilled in their safety net."