Business organizations joined civil rights activists and members of Congress yesterday in challenging a draft executive order that would abolish rules requiring some government contractors to meet numerical goals in hiring minorities and women.

White House aides attempted to play down the proposal, which several top officials, led by Attorney General Edwin Meese III and Assistant Attorney General William Bradford Reynolds, have urged President Reagan to approve.

Presidential spokesman Larry Speakes said in California that the executive order was "simply a draft" that has "no standing as administration policy" and is "in the early stages of discussion."

Another administration faction, led by Labor Secretary William E. Brock, is pushing an alternative approach that would preserve numerical goals and timetables for minority recruitment and hiring but allow for more flexible means of enforcement, according to sources.

The draft executive order, leaked by opponents Wednesday, would not require government contractors "to utilize any numerical quota, goal or ratio" to remedy racial or sexual discrimination. No contractor could be penalized for "failure to adopt or attain any statistical measures," regardless of the number of minorities and women hired, according to the document.

Critics noted that numerous court decisions have endorsed the use of numerical goals in discrimination cases. In a 1977 case involving the Teamsters union, the Supreme Court said in a unanimous ruling: "We have repeatedly approved the use of statistical proof . . . to establish a prima facie case of racial discrimination in jury selection cases . . . . Statistics are equally competent in proving employment discrimination."

Many of those covered by the current executive order signed by President Lyndon B. Johnson in 1965 -- more than 73,000 offices, plants and colleges with 23 million employes -- appear satisified with the current rules.

"We believe that using numerical goals and timetables is something that business is comfortable with, something that has been a proven success over the last two decades and something that business would like to continue to use," James Conway, associate director of equal opportunity for the National Association of Manufacturers, said. "Overall, we think it has worked."

Conway said some of the association's 13,600 member companies believe that the Labor Department's Office of Federal Contract Compliance Programs has been too rigid in enforcing the affirmative action rules, which can lead to a cutoff of federal contracts. But he said most regard numerical hiring goals as "a way of measuring progress."

In one study cited by the association, the proportion of black professionals in the work force jumped 72 percent during the 1970s, while women in professional jobs increased 53 percent.

"I don't think there's any question that affirmative action is good business and good public relations," said Lorence Kessler of the Equal Employment Advisory Council, a private trade group representing major government contractors. Kessler said many companies "have been able to use numerical goals without those goals becoming quotas," but some believe that federal oversight is too cumbersome.

The draft order is the latest round in a long debate over U.S. rules that give the Labor Department the power to force contractors to recruit, hire, train and promote more minorities and women.

The Reagan White House has previously tried to alter these rules, and critics say enforcement efforts have dropped off since the end of the Carter administration. A Labor Department official said that while voluntary compliance is now emphasized, both discrimination reviews and financial settlements in 1984 were double those of 1980.

In Santa Barbara, Calif., White House spokesman Speakes told reporters: "The president has indicated many times he opposes discrimination in all forms, including the practices of government contractors. At the same time, the president opposes numerical quotas as a means for achieving equality, and we support positive affirmative action programs in government."

The proposal drew strong support from conservatives. "Numerical goals are just another way of saying quotas, which is an improper and unconstitutional method," Paul Kamenar, executive legal director of the Washington Legal Foundation, said. "The Labor Department has been using that as a club that they call 'voluntary compliance,' which forces employers to engage in discrimination by following these race-conscious remedies or risk losing government contracts."

But civil rights supporters were angry. Nancy Kreiter of Women Employed, a Chicago-based advocacy group, called the proposal "a horrible retrenchment in civil rights . . . . I consider this a declaration of war by the Reagan administration in the area of equal opportunity."

Ralph G. Neas of the Leadership Conference on Civil Rights called it "a pernicious proposal" being pushed by administration "extremists."

Hyman Bookbinder of the American Jewish Committee said the plan would have "a chilling effect on affirmative action."

Sen. Howard M. Metzenbaum (D-Ohio), calling the draft order "shockingly disappointing," said: "I cannot understand Meese and Reynolds trying to sell this thing to the president. I don't understand where they're coming from.

"When you make an effort to determine whether there's been discrimination, you have to use whatever evidence is available," Metzenbaum said. "Doing away with the ability to use statistical data is tantamount to making it almost impossible to make a case."