A brother and sister who had brought a civil suit against their stepmother, claiming she was responsible for the May 1979 shooting death of their father, have settled out of court for $12,000.

Under the agreement, which was filed in D.C. Superior Court Monday and does not contain any finding of fault, Irene M. Thomas, 44, and Chester J. Thomas III, 41, will receive one-third of their father's $37,000 insurance policy. They originally had sought $1.5 million in damages.

The father, Chester J. Thomas Jr., a retired Navy Department employe, was killed on May 20, 1979, at his Northwest Washington home. Police have ruled the death a homicide. No arrests have been made in the case.

The stepmother, Catherine B. Thomas, denied in court papers that she was responsible for her husband's death. Her lawyer claimed that she "is entitled to her rightful share of the estate," which included all of the insurance policy.

Thomas' children, in their wrongful death suit, contended principally that Catherine Thomas had caused them "physical and mental pain and suffering" because of the loss of their father.

Their lawyer, Martin J. McNamara, alleged in court papers that Catherine Thomas, a retired medical officer with the Surgeon General's office, had threatened numerous times to kill her husband and that she "intentionally caused the decedent's death, either acting alone or in concert with one or more persons who remain unknown at present."

In court papers, Catherine Thomas said that she lived alone with her husband at 229 Ingraham St. NW and that she was there the night of the shooting. In a deposition taken in the case, she invoked the Fifth Amendment when asked by the children's attorneys to furnish details.

The junior Thomas, an employe with the Federal Home Loan Bank Board, and his sister, a $50,000-a-year community adviser with the Department of Housing and Urban Development, alleged in court papers that Catherine Thomas had threatened to kill her husband by various means, including "blowing up the garage with him in it."

"She said on many different occasions that she could put poison into his food or administer certain medications to my father that would kill him and no one would know that she was to blame," the son said in court papers.

McNamara said yesterday that the children, born from a previous marriage that ended in divorce, brought suit against their stepmother because "they felt there was a responsibility of some degree," and that they had agreed to the out-of-court settlement because they felt they had made their point.

"It's been rather traumatic," McNamara said. "We're glad to have it over."

Leonard L. McCants, attorney for Catherine Thomas, said last night that he wanted "to make sure it's clear that the settlement does not in any way indicate any culpability on Catherine Thomas' part . . . ." Nor, he said, did it indicate that any of the allegations made against her were even remotely true.

He said she settled "because the anxiety, time and expense connected with the suit were becoming too much" for her in terms of physical and mental wear and tear.