He was never very poplular with the powers that be, says Robert Kassebaum. After all, finding poor people to spend welfare money on is not a job that leaders of a rural, conservative county want vigorously pursued.

So when Kassebaum's love life became the main topic of gossip in the brick-tiled halls of the 140-year-old Caroline County Courthouse, the 53-year-old welfare director expected some flak. He didn't expect to be fired.

"I was railroaded out of my job," says Kassebaum, who was ousted in August by the unanimous vote of a four-member county welfare board.

A few weeks later, Rebecca Moore, a 34-year-old welfare agency worker who has been romantically linked to Kassebaum, also was fired. Kassebaum and Moore, who are separated from their spouses, were charged with bringing "discredit to the agency."

The controversy, which many say illustrates the strong -- and enduring -- influence of conventional moral values in Virginia's rural areas, has led to petitions in three courts and a death threat against Kasebaum. It threatens to revive a long-smoldering dispute over the future of Caroline, a bucolic county 65 miles south of Washington, where unemployment is among the highest in the state and more than half the residents must leave the county to work each day.

It has also generated enough local gossip to keep a soap opera forever in bubbles. "It's like our own 'Dallas,'" says one longtime resident of Caroline, which has remained steadfastly rural despite its prime location halfway between Washington and Richmond."This is such a small county, everybody knows about everything."

Kassebaum, a former Defense Department analyst who has worked in the Virginia welfare system for the past 14 years, maintains that it was not morals but politics that motivated his firing. An outspoken liberal, Kassebaum contends he was too conscientious about helping the poor.

In his six years as head of the county welfare system, he boosted the number of families on the politically unpopular aid-to-dependent-children program by 300 percent, an action that he says did little for his standing in Caroline. "The main concern of the people who run things in this county is to keep the tax rate down," says Kassebaum.

The members of the welfare board that fired him disagree. "No comment has ever been raised about the size of the [welfare] caseload," says welfare board head John G. Castles, who is also vice chairman of the county supervisors. Castles told local reporters in August that Kassebaum was not a good manager and complained that the 17-member welfare agency was beset by morale problems.

Kassebaum does not deny that there were disputes among his employes. But he blamed Castles' "repeated, irresponsible, direct intrusion into personnel matters" for causing "havoc in efforts to carry out sound administrative policies."

"His interest extends as far as the tax levied on his property," says Kassebaum of Castles, a brigadier general in the Virginia National Guard and the owner of a 270-acre estate with a swimming pool, riding ring and stables. "He makes all judgments based on the effect it's going to have on his real estate."

Castles denies that his holdings have influenced his decisions on either board and refuses to respond to Kassebaum's specific charges.

"I'm perfectly happy to let my reputation hang on my past performance," said Castles this week.

Kassebaum, who is white, claims to have the support of blacks who make up 51 percent of Caroline's 18,000 residents. But so far, black leaders have been reluctant to take sides in a confusing battle with such strong moral overtones.

"I don't know enough about the particulars to comment," says one of the two black supervisors on the county board, Luther Morris, who, like Kassebaum, has been a proponent of more industrial development in the county. "Caroline County is a very conservative county and Mr. Kassebaum is considered more of a liberal. He tried very much to get programs to the people and that may have been a shortfall of his," Morris said.

But the two black members of the welfare board deny that Kassebaum's firing had anything to do with his political persuasion. Both Richard Tillman and Edward Nelson say Kassebaum's romantic relationship with one of his employes created an untenable situation.

"I happen to feel that a director of welfare does have a moral responsibility," said Nelson this week. Kassebaum counters that Castles and the welfare board had no right to apply their moral standards to a private matter "that had nothing to do whatsoever with my performance as director."

While welfare board members complain about Kassebaum's performance as a manager, Kassebaum claims Caroline has the best-run welfare agency in Virginia. One indication of that, says Kassebaum, is that for the quarter that ended in June, Caroline had the lowest proportional administrative cost of any welfare agency in Northern Virginia.

Virginia welfare officials confirm Kassebaum's figures, but they caution those figures are just one measure of good management. The department has otherwise remained outside the controversy. Although the commonwealth's 124 local welfare directors are officers of the state system, which reimburses localities for 80 percent of their annual salaries, they are considered local employes.

One state welfare official familiar with the Caroline controversy rates Kassebaum as a better-than-average welfare director.

"The county welfare director has a very difficult role to play," said the official, who asked not to be identified. "He has to please the county board which employs him at the same time that he carries out his state and federal requirements which do not always please those county boards. He has to walk a very delicate line."