The Carter administration yesterday signed a sweeping affirmative action agreement designed to guarantee minorities a larger share of better-paying federal jobs.

Associate Attorney General John H. Shenefield told reporters he had turned down a request by Reagan transition team official Loren Smith to hold off signing the agreement and leave the matter for the Reagan administration to decide.

The agreement, controversial because some government lawyers and other observers think it sets up a quota system, took the form of a consent decree to settle a minority lawsuit against the government. Shenefield, who said he does not regard it as establishing quotas, submitted the decree to U.S. District Court Judge Joyce H. Green yesterday, his last day in office.

The decree will not become final until approved by Green, who took it under advisement. Spokesmen for the Reagan transition team appeared in court and asked her not to act until they could submit their views. The decree allows 60 days for anyone interested to comment to the judge.

Under the agreement, reached after almost two years of negotiation, the government would junk the Professional and Administrative Career Examination (PACE) by Jan. 1, 1984. PACE, a written competency test, is the most important civil service exam. But it has been criticized as favoring whites over blacks and Hispanics in determining who gets jobs at the GS5 and GS7 entry level.

In its place, federal agencies would be required to construct new tests, designed to guarantee that a higher proportion of black and Hispanic candidates would pass and get jobs.

The tests would seek to assure that the number of blacks and Hispanics getting jobs would be proportional to the number who took them. For example, if half taking an exam for a job were blacks and Hispanics, then about half (but no less than 40 percent) of them would wind up with jobs.

If the new test did not produce the required percentages of blacks and Hispanics, the agency then would have several alternatives.

It could junk the new test and make up another one. Or it could seek to "validate" the test; that is, try to demonstrtate it contained no hidden biases and is an excellent test for obtaining the best qualified employes. At the same time, it would have to take special steps to recruit minority candidates without requiring them to take any test.

There are a number of special outreach programs spelled out in the consent decree. The outstanding scholar program allows hiring of college students with a grade average of at least 3.5 (on a 4.0 scale). Another allows hiring of students who do well in government work-study programs. And another gives preference to applicants with bilingual skills.

A number of government attorneys involved in the case said the agreement amounts to a quota system. But Shenefield said a quota imposes an absolute numercial requirement. The aim of the agreement, he said, is to find qualified people, including more qualified blacks and Hispanics.

He argued that under the agreement if any agency has validated an exam and is using special outreach efforts, it is not obligated to rewrite the exam even if it is not producing the desired proportion of blacks and Hispanics.

However, other attorneys said there is nothing to stop a disappointed applicant from going to court and charging that even a written exam contains hidden biases.

Nathan Perlmutter, national director of the Anti-Defamation League of B'nai B'rith, said he views the arrangement as a quota system, which he called "a mischievous nostrum for the problem of minority unemployment."

However, the Mexican American Legal Defense and Educational Fund said the agreement would reverse "longstanding discrimination."

In 1979, a total of 137,725 persons took the PACE exam for jobs like customs inspector, immigration officer and revenue officer, and 6,283 were selected. On Jan. 29, 1979, a group of blacks and Hispanics who had failed the April 1978 exam sued the Office of Personnel Management, alleging PACE contained hidden biases that excluded them from jobs.

Figures showed that 42 percent of the whites taking the test got a passing grade of 70 percent or more, but only 5 percent of the blacks and 13 percent of the Hispanics. The plaintiffs said the test demanded knowledge not really needed to fill the 118 categories of jobs for which it was used.