In a jammed Montgomery County courtroom last week, prosecutors rose to make a legal point, bolstering their case against Edward Thomas Mann. Suddenly, the defendant, charged with murdering three people in a rampage at IBM offices last spring, swiveled in his chair and faced his accusers.

"Thank you," he said quietly and politely.

It was an incongruous moment--with the accused warmly complimenting his accusers--but it was just one of many such moments since last May when Mann was charged with murder. Since that time, Mann has hired, then fired, one of Washington's foremost criminal defense attorneys. He has changed his plea of innocent by reason of insanity to a plea of guilty. He has talked of a "conspiracy" to destroy him that includes a plot to force him to pick up male and female prostitutes. He has attempted suicide at least once and has told the judge that he wants to be executed.

But Montgomery County Circuit Court Judge William C. Miller, noting Mann's bizarre behavior in recent months, has refused to accept the guilty plea. Instead, Miller is conducting hearings to determine whether Mann is rational and should go on trial for his alleged crimes.

The 38-year-old defendant is accused of ramming his Lincoln Continental through the glass lobby of the IBM building in Bethesda, where he once worked, and then shooting wildly with an automatic weapon, killing three persons.

The hearings, held last week and scheduled to resume Feb. 7, have nothing to do with Mann's guilt or innocence. Nor is the judge concerned right now with the question of whether Mann was insane at the time of the shootings, as Mann's former defense attorney maintained.

Instead, in this extremely rare type of hearing, Judge Miller is addressing the complex question of whether Mann is mentally competent to stand trial, an issue based on whether Mann can understand the legal proceedings and assist in his own defense.

If Mann is found "incompetent," as the defense contends he is, he will return to a state mental hospital, where he would come up for periodic competency reviews. If his condition changes, he could be tried in the future. But if he is not found competent within 10 years, the court could dismiss the charges against him and set him free. Or the court could find that he is "dangerous" to himself or others and commit him to a state mental hospital.

Mann's attorneys are seeking the finding of incompetency, because they fear, to use a phrase Judge Miller coined, that Mann is trying to commit "judicial suicide."

In the competency hearings, Miller is faced with three main issues: Is Mann mentally fit to stand trial? Can Mann, who everyone agrees is suffering from a mental disorder, be allowed to act as his own attorney? And can he be allowed to throw away his constitutional right to a trial and plead guilty to murder?

Miller's task is not an easy one. He is faced with a family history full of severe mental disorders--Mann's brother and great-grandfather committed suicide, his mother suffered from schizophrenia and his father may have suffered from paranoia, according to court testimony.

In addition, Miller said he wonders if Mann is seeking the kind of perverse stardom that murderer Gary Gilmore achieved when he insisted upon being shot to death and became the first American executed in nearly a decade.

It is these sorts of issues that dramatize the conflicts between law and psychiatry when it comes to determining a defendant's mental fitness.

"A person can be severely mentally ill, even psychotic, and can still be competent to stand trial," said lawyer Bruce Ennis, who is chairman of the American Bar Association's Commission on the Mentally Disabled.

During four days in the courtroom last week, Miller heard conflicting testimony from seven psychiatrists and psychologists. All the experts agreed on two points, that Mann is intelligent and that he is suffering from some type of paranoia, the feeling that his former employer, IBM, is conspiring to destroy his life. But they could not agree--and this is the crucial question--on whether he also believes that his defense attorneys and the judge are part of that plot.

The prosecutors argued that because of Mann's intelligence and his ability in the past months to file certain legal papers, he is, indeed, competent to understand the court proceedings. His paranoia, they insisted, has nothing to do with this ability. It is the prosecutors' job to prove that Mann is competent beyond a reasonable doubt.

The defense, however, has brought out testimony from the experts that Mann believes the court is part of the conspiracy against him and that Mann wants to use the court proceedings to expose IBM. If that is true, the judge will have to determine if it is rational for Mann to see the court, and perhaps, his own execution as ways to expose IBM's alleged injustice.

Miller was faced last week with a defendant, calm and polite in a blue pin-striped suit, who questioned witnesses with a display of humor and incisiveness. But testimony also portrayed a man whose actions and beliefs call his mental state into question.

According to some of the experts, Mann believes that IBM has spearheaded a "global conspiracy" against him ever since he left his job as a computer salesman in 1979. Mann feels that IBM and others involved in the plot have planted messages in newspapers to manipulate him, put poison in his mouthwash and used his only son as a pawn to prod him into fights.

After his arrest, Mann hired one of Washington's foremost experts on the insanity defense, attorney R. Kenneth Mundy. Then he complained that Mundy wouldn't allow him to hold press conferences or file the legal motions he desired, and he fired him.

Mann also tried to fire public defenders, temporarily appointed by the judge to represent him in the competency hearing. In court he shuns these attorneys.

During the holidays, Mann sent Judge Miller an elaborate Christmas card, parodying the behavior of the judge and one of the public defenders. Mann mentioned the card in court last week. "I realize you may not have liked my Christmas card," Mann told the judge. "But can you remove these shackles from my legs?"

On Jan. 6, the day Mann was originally scheduled to begin the competency hearings, he punched his hand through a television set at the Montgomery County Detention Center and allegedly swallowed cleaning fluid in a suicide attempt.

Miller will have to weigh all this as he tries to determine whether Mann is competent to stand trial. But even if he finds Mann competent, there are still the questions of whether Mann can act as his own attorney and be allowed to plead guilty.

"The general feeling seems to be developing that competency to plead guilty requires a different and higher level of mental ability," said Ennis, an expert on questions of legal competency. When a man is found competent to stand trial, it means that he will have his day in court, Ennis explained. But in pleading guilty, a defendant "is waiving all kinds of constitutional rights," including the right to testify and to have a jury determine guilt or innocence.

The standard for competency to plead guilty, according to recent appellate decisions, has become the ability to freely make a "reasoned choice," and to understand its consequences unimpaired by mental disability.

"These decisions fall into the no-man's-land between legal and psychiatric concepts," said Ennis. "Therefore, there is no clear guideline to assist judges in resolving them."