The Abscam conviction of former Florida Republican congressman Richard Kelly, who was videotaped stuffing $100 bills into his pockets, was overturned here yesterday by U.S. District Court Judge William B. Bryant, who said Kelly's rights were violated.

"Law enforcement exceeds its bounds when it manufactures crimes and creates criminals," Bryant said in one of the strongest criticisms yet of the controversial and unprecedented FBI investigation. "A person corrupted under circumstances which only police officials can create or by a process which only the authorities are licensed to use, has been made into a criminal by his own government."

Entrapment has been one of the key issues in the Abscam cases since their inception. The question of how far the government could go in setting up situations that might cause public officials to break the law has been at the center of many of the Abscam appeals.

Almost all of the Abscam defendants claimed in their defenses that they were entrapped by government agents, and it is an issue that is not likely to be settled until the Supreme Court rules on one of the cases.

Bryant, often using strong language, said the government had no reason to believe that Kelly had any predisposition toward criminal activity and that Kelly turned down the payoff offer three times before he finally took the money in front of the hidden cameras.

Kelly, defeated in a reelection bid in the 1980 primary, was convicted of bribery and conspiracy Jan. 26, 1981. He has been free on bond pending Bryant's decision on a motion to dismiss the indictment and the jury's conviction.

In St. Petersburg, Fla., the Associated Press said Kelly seemed somber when he met with reporters. The FBI, Kelly said, "did a super job of snookering everybody."

"The most astounding thing is that they simply didn't have any evidence at all . . . . It was all created by the FBI . . . . I'm really kind of a bomb around here because I'm not feeling very elated. I'm not feeling very happy about it."

Kelly was one of seven members of Congress and a number of lower level officials convicted in the controversial investigation that featured FBI agents posing as phony Arab shieks offering big money for legislative favors.

The Justice Department said Bryant's ruling would be appealed.

Bryant said repeatedly that Kelly's crime was manufactured by the government and that his constitutional rights to due process were violated.

"Government agents, hard about the business of corrupting public officials who are free of suspicion, essentially subvert our government; and on its face this presents an unwholesome spectacle," Bryant said, calling the government behavior "so outrageous that it transcends any standard of fundamental fairness."

The judge said the government behavior in the Kelly case "creates a whole new type of crime that would not exist but for the government's actions. When improper proposals are rejected in these virtue-testing ventures, the guinea pig should be left alone."

He said Abscam was not a traditional "sting" operation in which agents pose as criminals because it used a "recruiting agent who was programmed to go out" and lure members of Congress to a supposedly legitimate business discussion.

"Where the government's overtures to the defendant fail substantially to model real-world behavior, the results of government conduct, while satisfying the technical elements of the bribery statute, constitute a crime which would never otherwise occur," he said.

"I do not believe that testing virtue is a function of law enforcement. But this personal belief aside . . . the method of testing must be fair. If after an illegal offer is made, the subject rejects it in any fashion, the government cannot press on."

Bryant, who presided over the jury that convicted Kelly in 1981, had said during that trial that the case "has an odor to it that is going to be cleared away before anybody gets convicted. It has an odor to it that is absolutely repulsive. Let's get along with the trial . . . . But it stinks."

In addition to dropping the indictment against Kelly, Bryant also granted new trials to two men convicted with Kelly: Eugene (Gino) Ciuzio, a Longwood, Fla., businessman, and Stanley Weisz, an accountant from Smithtown, N.Y. The government said both were middlemen in a bribery scheme in which Kelly allegedly was involved.

Before and during his trial, Kelly maintained that he took the $25,000 payoff--which he returned to the government after the Abscam investigation became public--as part of his own secret investigation into suspicious characters he thought had infiltrated his congressional office.

He claimed he believed these characters were determined to destroy him politically. But he did not go to the FBI.

In his order, Bryant characterizes Kelly's claim that he was conducting his own investigation as "bizarre, nearly farcical . . . . Not a scintilla of evidence which would support this theory ever surfaced in this case."

However, Bryant continued, "As the case unfolded, I was plagued with the unsettling realization that with this loose cannon rolling around on the deck of the criminal justice system, such a thing could indeed occur."

In the trial, Weisz was shown on videotape taking $50,000, which he said he regarded as a finder's fee for introducing Kelly to representatives of the phony sheik. Ciuzio, the other codefendant, said he took part because he was trying to outwit the sheik's "con men."

Kelly was the only Republican among six House members convicted of Abscam charges. The only senator convicted in Abscam was Harrison A. Williams Jr. of New Jersey.

Major questions about the conduct of the FBI in the Abscam investigation have been raised. Rep. Don Edwards (D-Calif.), chairman of a House Judiciary subcommittee, has been holding hearings on the matter off and on for nearly two years. He said yesterday that the Bryant ruling "is an indication the work we have been doing and will continue to do is on a firm foundation."

In the past month, the Senate has also been organizing a select committee to investigate Abscam.

FBI Director William H. Webster has repeatedly defended his bureau's behavior in the matter, pointing out that those who took bribes have been convicted.

In 1980, a trial court judge in Pennsylvania overruled a jury's guilty verdicts against two Philadelphia city councilmen who had been recorded on videotapes taking cash from undercover FBI agents.

But the U.S. 3rd Circuit Court of Appeals reinstated those convictions last February. The two men have appealed that finding to the Supreme Court.

It is not yet clear how Bryant's ruling will affect the various Abscam appeals that are under way. Bob Flynn, attorney for Williams, said of the Bryant ruling, "It's got to help."