Betty Margoles remembers vividly the warm July day in 1960, with the radio news blaring from a transistor radio as she worked on her flower beds in an affluent Milwaukee suburb.

Suddenly, the newscaster was reporting that the FBI was surrounding the hospital where Dr. Milton Margoles was performing surgery, that her husband had been charged with attempting to bribe a federal judge, and that he was trying to escape.

As the Margoles family looks back over events, they see that day as the first chilling sign of what was to come--that what had started out as a relatively simple tax violation was about to grow into a strange and unending tangle of problems with authorities, both federal and state, that would consume their lives from that day forward.

The Margoles story sounds impossible and bizarre. It sounds like the product of a paranoid mind. Even family members find parts of it hard to believe. They call it their "own private Russian novel."

They seem to be a family trapped by unknown forces in a society where this sort of thing is not supposed to happen, where law is intended to protect rather than to harass.

But even though their claims are difficult to believe, the Margoles family surrounds itself with file drawer after file drawer of documents to back them up. They have spent the past 21 years gathering these documents from public records, from court suits and through freedom of information requests.

Ultimately, the Margoles case has involved a presidential pardon, rumors of illegal abortions, congressional hearings, allegations of crooked judges, conflicts of interest, questionable legal advice, an illegal break-in, illegal wiretapping and an effort by the Wisconsin medical licensing board to prevent Margoles from practicing medicine again in other states.

By the time the doctor finished paying off his debt to the Internal Revenue Service in 1974, the original $33,000 assessment had grown to more than $400,000 including interest and penalties, and he had paid more than $200,000 in legal fees.

By then, his son Perry Margoles had finished college and law school, and began to carry forward the family legal battle.

The family is still on shaky financial ground, but all three are reluctant to talk about it. "Pretenses become very important," Perry Margoles says. "You don't want them to know how bad things are because they might just push us that little bit more."

Today Margoles, now 69 years old and suffering from a serious heart ailment, is hoping to end the case before he dies. After being turned down on four different issues by the Supreme Court, the family plans one last attempt this month. They are filing papers asking the Supreme Court to overturn the finding of a judge they claim was biased.

The litigation has virtually consumed the lives of the Margoles family, including that of Perry, now 37, who was 16 when his father went to prison. Since he finished law school in 1969, he has worked virtually full time on the lawsuits and on trying to keep the family afloat financially.

In contrast to the publicity of their life in Milwaukee, the family now lives largely in seclusion here in a wooden octagon-shaped house in the woods of northeastern Illinois. Their time is devoted almost solely to the case.

The Margoles story seems to have begun in 1946 when the doctor returned from the war, eager to resume medical practice in Milwaukee.

Margoles is Jewish, and at that time he says hospitals in the area were less than anxious to have him on their staffs. To make matters more difficult, he had a racially integrated practice.

If Milwaukee hospitals were more leery of anything than Jewish doctors at that time, Margoles said, it was black patients. And Margoles has saved a letter he received in 1947 from an area hospital saying he could practice there, but "due to the limited facilities, we cannot accommodate Negro patients."

Margoles decided to build his own integrated facility, Capitol Hospital, which allowed not only black patients, but also black doctors. He soon had the largest black practice in Milwaukee and a good income.

The family lived in a large, elegant house in one of Milwaukee's nicest neighborhoods, his son went to summer camp and Mrs. Margoles was active in Hadassah, the March of Dimes, school tutoring, Red Cross and the Urban League.

But according to Margoles, his success was generating considerable resentment in Milwaukee medical circles. And the hostility escalated in 1955 when an anesthetist fired by Capitol went to the Wisconsin State Board of Medical Examiners and charged not only that Margoles had performed an illegal abortion, but also that he had seen Margoles cut up a fetus of three months and flush it down a toilet.

Margoles says he was unaware of the abortion charge until 1965, after which he was able to prove that the patient named in the complaint had not only been operated on by a different doctor, but also that she was not even pregnant.

The board was not able to substantiate the abortion charge, and did not pursue it further. But the allegation stayed in the board's files. It was about that time that the IRS got into the case.

Margoles claims the medical board asked the service to investigate him. Board members have denied the charge in court.

But whatever its origin, in March, 1957, the doctor received a letter from the IRS saying he owed $33,716 in back taxes on unreported interest income. He says his attorneys advised him to fight.

Margoles now concedes he owed the money because of a misunderstanding on reporting interest income, and in retrospect he wishes he had paid at the time.

But in 1960, as interest and penalties continued to compound and the IRS continued to pursue him for tax evasion, his attorneys worked out a plea-bargaining arrangement under which they believed Margoles would receive a fine and suspended sentence in return for pleading "no contest."

The judge was the late Robert E. Tehan, of the U.S. District Court in Milwaukee, who was known for his leniency in tax cases. It would be more than another decade before the public would learn that Tehan himself had failed to file or pay federal and state income taxes for eight years before he went on the federal bench.

However, instead of going along with the plea-bargaining, Tehan announced that he knew the doctor had hidden $731,000 in secret assets. (The IRS finally conceded in 1969 that there was no basis for the charge.) Tehan sentenced Margoles to a year in jail, ordered him to pay the compounded taxes and added a $15,000 fine.

Margoles was given 90 days to get his affairs in order before reporting to prison. During that time, he was contacted by a man who said Margoles could get his sentence suspended if he would hire a local law firm which included Tehan's son as a member.

"I was suspicious. I went to see my lawyers, and they said go ahead. But it was a setup," Margoles said. He paid a $5,000 retainer to the firm and was immediately charged with attempted bribery. It was at that point the FBI went to Margoles' hospital to arrest him.

The lawyer who advised Margoles to pay the $5,000 is now dead. He did not testify at the trial, Margoles says, because he was afraid of retaliation the next time he brought a case before the judge. In 1965 the lawyer testified before the medical board in Illinois that he had advised the doctor to pay.

Margoles was acquitted of the bribery charge but convicted of obstruction of justice. He was fined $10,000 and sentenced to another five years in prison.

"Was it bad judgment? Sure it was," Perry Margoles says now. "But he wasn't a crook. You have to look at the thing in context--with them violating the plea-bargaining agreement, with the judge's son carrying on an active practice in his father's court."

But the Margoles family says that even though they were distressed, they could have dealt with what happened up to that point. What became unbearable were the events that followed.

First, there was an extortion attempt by an organized crime figure anxious to get part of the hidden assets Tehan had talked about.

After that was resolved, they discovered their house had been bugged, apparently by an IRS agent posing as a telephone company employe. They might never have found out except that when he returned in 1960 to break in and remove the listening devices, Mrs. Margoles called the police and had him arrested.

The man told police he had come to deliver a tax levy, but was taken to headquarters when he could not explain why he had parked two blocks away.

Margoles says a security firm hired by the family found evidence of three listening devices in the home.

In later lawsuits Margoles obtained two Justice Department memos concerning the wiretap. The first, dated Feb. 7, 1967, states that the IRS "has found that the wiretap was not connected with the merits of the taxpayer's liability, and only was used by an eager agent attempting to find assets for collection of the liability."

The memo goes on to admit that the agent got access to the house by posing as an employe of the telephone company.

But a second memo, on Feb. 20, 1967, the day of a hearing on the wiretapping, revised the earlier one, stating that the IRS "in actuality, found no evidence of wiretap in this case . . . . the only thing the service found in investigating his allegations was that a revenue agent had been overeager in attempting to find assets."

When Margoles got out of prison in 1962, his first thought was to resume his medical practice and to begin to pay off his mounting debt to the IRS.

But he discovered not only that his medical license had been revoked, but also that Capitol Hospital had gone into bankruptcy. He says all records of hospital funds, assets and accounts receivable had disappeared.

Margoles began to apply for licensing in other states, but wherever he went, he says his past continued to haunt him, not just the tax charges but also rumors about abortions.

It was not until 1965, at a reinstatement hearing before the Wisconsin board, that Margoles learned of the abortion charge. But he was still not given the date of the alleged abortion or the name of the patient, information that had been in the board's files for nearly a decade.

The anesthetist testified he could no longer remember the specifics, making it harder for Margoles to refute the charges. The tax violation was also raised, and the board refused to restore his license on grounds of his "moral character."

Margoles recalls getting a job offer in Lansing, Mich. After he had driven the 250 miles, he says he found a letter waiting at the Holiday Inn from the doctor who had made the offer. It said simply that the doctor had come across new information and no longer wanted to be associated with Margoles in any way.

Finally, he received accreditation in several states and practiced for a few years in Michigan, commuting to Milwaukee on weekends.

Meanwhile, by about 1969 the tiny community of Zion, Ill., just south of Winthrop Harbor and less than an hour from Milwaukee, began a campaign to recruit doctors and became interested in Margoles.

He applied for an Illinois medical license. But while his application was under consideration, members and representatives of the Wisconsin board communicated with Illinois officials, criticizing Margoles' character and performance.

Then, Alida Johns, a reporter for the Milwaukee Sentinel, confided to an Illinois official that Margoles was known as a "pathological liar," according to an affidavit filed by that official.

Rep. Robert McClory (R-Ill.), who represents Zion, had read about the Margoles case and in 1970 began to talk about the possibility of asking President Nixon for a pardon.

As McClory's interest was publicized, he says members of his staff began receiving calls from Johns. Affidavits from those employes indicate she accused Margoles of running an "abortion palace" and a "house of ill fame" and that he was "unfit for help by decent people."

In a later deposition, Johns stated she had not made the charges personally and that she had simply read a newspaper story about Margoles over the telephone to McClory's employes and the Illinois official.

Margoles filed a civil rights suit against Dr. Leroy Dalton and Thomas Tormey and a slander suit against Johns and her newspaper. He charged that Tormey and Dalton, both former representatives of the Wisconsin medical board, had violated the doctor's civil rights in sending the critical letters to the Illinois licensing board.

Margoles finally obtained his Illinois license in 1971 and settled down here to a small-town practice, hopeful that his life was back on the right track.

Late in 1972, the Margoles family thought they had finally been vindicated when Nixon granted the doctor a full pardon.

Then in 1974 McClory introduced a bill in Congress asking for $195,000 in compensation for Margoles because of the "arbitrary, negligent and unduly harsh methods" of the IRS, and Margoles came to Washington to tell his story to a House subcommittee. But the bill was never reported out of committee.

In December, 1975, Margoles had a serious heart attack.

Meanwhile, the family was going ahead with the civil rights and slander suits.

When a new judge was assigned, Margoles says he was "horrified" to discover it was Robert W. Warren, newly appointed to the bench, who just two weeks before had been attorney general of Wisconsin. Documents obtained by Margoles from the attorney general's office indicate Warren had been periodically apprised of developments in the civil rights case against Dalton and Tormey.

Margoles' attorney informally asked the judge to step aside, but he refused. Then on Jan. 5, 1976, shortly after Margoles suffered his heart attack, Warren scheduled a pretrial hearing. Perry Margoles says he asked his lawyer to request a continuance because his father was still in the hospital and that the lawyer told him the hearing had been postponed.

But it went ahead as scheduled, and Warren threw the case out of court on grouds that Margoles was willfully late in providing documents to the court.

Perry Margoles says that when documents were late, it was generally because his lawyer could not find the time to meet him, and he has filed a malpractice suit against the lawyer.

Margoles finally recovered and resumed practice, but he says he was still suspicious that something was going on. He contacted the Illinois medical board and asked them to notify him of any complaints.

He noticed a stream of new patients coming into his office asking for tranquilizers and diet pills.

It was exactly what he feared. In May, 1978, a 37-count indictment was filed, accusing Margoles of illegally prescribing drugs to government agents posing as patients. Margoles insists he did not prescribe anything that was not warranted by the symptoms described by the phony patients.

He soon began to experience new heart problems and by September he was undergoing open-heart surgery.

Finally, in December, 1979, federal attorneys dropped plans for prosecution and Margoles agreed to sign a form saying the drugs may have been used for "other than therapeutic" purposes.

But just a week later, Margoles was back in court on his civil rights case against Dalton and Tormey.

The jury ruled that Tormey and Dalton had violated his civil rights.

But before the jury could meet again to award damages, on Jan. 10, 1980, U.S. District Court Judge Barbara Crabb, who had presided, reversed the jury decision saying, "The evidence in this case cannot support the conclusion that the defendants violated any constitutional rights of Dr. Margoles."

Most of Margoles's cases have been appealed as far as they can go.

The final act will begin to unfold this month at the Supreme Court. Margoles will ask the court to review the dismissal of the slander suit against Johns. Margoles is charging that Warren had previous knowledge of the case and should have stepped aside.

The Margoles family understands that their chances are slim, that the court can hear only a fraction of the cases that are submitted.

No one will say exactly what their plans are once the ordeal is over. The question makes Perry Margoles uneasy. "We don't look ahead more than several weeks at a time," he said. "Maybe if this ever ends, maybe I'll be able to get a daughter-in-law and some grandkids for them.