Two months ago, doctors at St. Elizabeths Hospital sent a letter to a D.C. Superior Court judge recommending that he authorize them to discharge outpatient Ernest L. Dorsey because Dorsey was no longer a threat to himself or others.

Last week, before the judge who ordered him confined there five years ago had ruled on the doctors' recommendation, District police arrested the 33-year-old Dorsey.

The charge: two counts each of rape and robbery. The victims: two St. Elizabeths employes.

Dorsey, who was admitted to the hospital in 1976 after he was found innocent -- by reason of insanity-- on a charge of assault with a deadly weapon, also is being investigated in connection with another rape and robbery involving two other hospital employes, police said.

According to police, the rapes and robberies with which Dorsey is charged occurred last Wednesday. Police say that in those incidents a man entered a building on the hospital grounds, pointed a gun at two women and forced them to leave the building. They were then forced to drive the man to a nearby check-cashing service.

There, police said, the man ordered the two women to cash their paychecks and give the money to him. The women were then forced to drive him back to the hospital grounds where they were led into a basement bathroom and raped, police said.

Subsequently, the man led the two back to the car and directed them to drive to a liquor store, police said. Once inside the store, however, one of the victims began struggling with the man, who fled when the store owner came to the woman's aid.

The day after that, two other female employes in another building on the hospital grounds were robbed by a man who tied up both women, police said. The man then raped one of the women and took a set of keys, police said.

According to Harold Thomas, special assistant to the superintendent of St. Elizabeths, Dorsey's was a familiar face to hospital staff. Despite the fact that he had been an outpatient there since February, he spent most of his time at the hospital. Until two weeks ago, Thomas said, Dorsey had worked on the bakery truck on the hospital grounds.

"Although he lived in the city, this was his home," Thomas said. "When he didn't have an appointment, he would show up and hang around the grounds. He was not a stranger. He pretty much knew the staff movement."

Thomas said hospital authorities recommended to the court in June that Dorsey be released entirely from the hospital because "he had received maximum benefit from hospitalization." The judge had taken the hospital's recommendation under advisement and was expected to rule in the matter in September, he said.

Dorsey's arrest on Friday at 122 R St. NE culminated a three-day period in which female hospital employes, alarmed by the incidents, had begun traveling in groups from place to place on the hospital grounds for safety reasons.

"Nobody wants to go anywhere alone," one hospital cafeteria worker said yesterday.

Thomas said the rapes have prompted hospital authorities totighten security. They are also studying whether to install various security systems, including the buzzers on doors and other electronic devices, he said.

Dr. Valarie Lathrop, deputy director for nursing, said the staff had been alerted to be more cautious following the rapes.

"We all realize that this is a special type of situation that the best security would not be able to prevent," she said.

Lathrop said hospital officials want to maintain as much free access to buildings on the hospital grounds as possible in order to keep the atmosphere closer to that of a hospital than a jail.

But she noted that "you are in some ways caught between a rock and a hard place in terms of how much security is enough."

Dorsey was being confined in a maximum security section of the hospital pending the outcome of court action on the new charges against him.