A federal judge ruled today that a technical error by President Carter in 1980 invalidates the Selective Service draft registration system. The decision stunned attorneys and gave the anti-war movement a significant victory.

The ruling by U.S. District Court Judge Terry J. Hatter Jr. came as he dismissed a case against former Yale University philosophy student David Wayte on grounds that Carter ordered the system into effect after waiting only 21 of the 30 days required from publication of the new law in the Federal Register. Wayte was charged with failure to register for Selective Service.

Hatter also said available evidence indicated that Wayte had been a victim of unconstitutional "selective prosecution" because he was indicted only after he publicly declared his refusal to sign a registration card.

"We do have a legal problem," a Justice Department spokesman said in Washington tonight. He said that the ruling would be appealed immediately to the 9th U.S. Circuit Court of Appeals and that no further comment would be made until the department reads Hatter's decision.

American Civil Liberties Union attorney Mark Rosenbaum, who represented Wayte, said he was "pleasantly stunned" by the decision.

The ruling could pose a major obstacle to the registration program by giving other outspoken resisters an important precedent and easing any fear of punishment among the estimated 500,000 young men who have failed to register and not identified themselves.

Wayte, 21, a Pasadena resident, was elated after the decision. "I think it was a big victory for us and a setback for the Selective Service System," he said.

He had contended that, by registering, he would only make it easier for the government to wage unnecessary wars.

After announcing his resistance at a 1980 press conference in San Francisco and at a demonstration in Santa Cruz, Wayte received an official warning from the government that he would be indicted if he did not fill out a card.

At least 13 men have been indicted for violating the registration law, which requires American men to fill out a card at their local post office when they turn 18 so they can be quickly reached in the event the military draft is reinstated.

The first three men tried under the law were convicted. One was sentenced to probation if he agreed to register, one was fined $4,000 and ordered to work two years with the mentally retarded, and the third was sentenced to 30 months in prison. Their cases are under appeal.

Bill Smith, one of Wayte's attorneys, said recently that there had been no new indictments for several weeks and suggested that the government was awaiting the outcome of Wayte's case.

Earlier, Hatter had ordered the government to give Wayte's attorneys certain internal White House and Selective Service System documents and bring White House counselor Edwin Meese III to testify on the initial decisions about who would be prosecuted.

Assistant U.S. Attorney Richard Romero, representing the government, refused and claimed that a form of executive privilege protects the documents and Meese.

The government invited Hatter to dismiss the case so that the 9th U.S. Circuit Court of Appeals could decide the issue.

The Los Angeles Times reported tonight that the government, apparently moving to show it also intends to prosecute non-registrants who did not voluntarily disclose themselves, had begun its "active enforcement" program.

The Times said the Selective Service System had sent the Justice Department the names of five men whose failure to register was determined in a computer matching of Selective Service lists with those of the 18-to-22-year age category on Social Security rolls.

Internal Revenue Service records are being used as the best available source of the non-registrants' addresses, the Times reported.