W. Jarvis Moody resigned yesterday as chairman of American Security Corp. as the District's second-largest bank company reported that it lost $11.9 million in the final three months of 1984.

Moody, 56, was immediately replaced by Daniel J. Callahan, who was president of rival Riggs National Bank until he became American Security's president in 1983.

Moody, whose resignation was unexpected, cited health reasons for his decision at a special meeting of American Security directors yesterday.

Bank sources said Moody was not forced out. They said he decided to to resign because of the pressures he would face in coming years trying to right American Security, whose loan portfolio is riddled with problems. Moody presumably has been under extraordinary stress since last spring, when loan losses began to mount.

A spokesman for American Security said that "according to doctors, the health problems referred to are best remedied by a temporary tapering off of the demanding pace required of a regional bank holding company chairman." But American Security officials said they could not specify Moody's health problems.

Unlike Moody's resignation, the bank's $11.9 million fourth-quarter loss ($1.04 a share) did not come as a surprise. Following an examination by federal regulators early last month, the big bank company announced that it had agreed to make a $37 million addition to its reserves as a cushion against possible loan losses.

American Security, whose assets total $4.1 billion, said then it expected to report a fourth-quarter loss and only a small profit for the year. Additions to loan loss reserves are substracted directly from earnings.

The bank company reported yesterday that for all of 1984 it earned $4.7 million (41 cents) compared with $31.4 million ($2.81) in 1983. In the 1983 fourth quarter, the bank's profits were $7.9 million (70 cents).

The bank said yesterday it has pledged to the Comptroller of the Currency that it will take additional steps to strengthen itself.

The bank agreed to stiffen policies for evaluating new loans and analyzing credits already on the books, promised to review personnel responsible for making loans and to maintain adequate reserves to cover potential loan losses.

As a result of the $37 million loan loss reserve addition taken last month, analysts and other Washington bankers said that American Security seems to have more than enough cushion to cover loan losses.

Even though the bank is financially stable, they said, it will be some time before it can return to its traditional profitability because its long list of problem loans will be a major drag on its earning power.

American Security reported yesterday that it has $145 million of problem loans -- most of them loans that are overdue by 90 days or more. They account for 6.6 percent of the bank's total loans. About 40 percent, or $54 million, of those problem loans are concentrated in real estate and energy. Another $10 million are maritime loans and $22 million are international loans.

Reducing that level of problem loans will be a gradual and difficult process. In some cases the loans will be written off -- charged against the reserve that American Security built up at the behest of the comptroller's office. In other cases, the bank may have to foreclose and sell the collateral. In most cases, problem loans return to health -- but the process generally takes time.

Callahan, who was elected chairman of both the parent company and the bank -- and will remain president of the bank -- said in an interview that he believes "this bank is going to recover and be back in a top posture in a reasonably short period of time. We have a solid base of customers -- both depositors and borrowers."

Callahan said that American Security is likely to shift its lending focus back to the Washington region in coming years. The bank began to open loan offices around the country several years ago, but already has closed two of the seven, including the New Orleans office, which made the problem energy loans.

Until 1981, when Joe L. Allbritton bought control of Riggs, it had been generally assumed that Callahan, 52, would eventually succeed Vincent C. Burke Jr. as chairman of Washington's biggest bank. But Allbritton forced out Burke in 1982 and Callahan, reportedly unhappy because he had few responsibilities, moved to American Security in May 1983.

Moody, who joined American Security two decades ago after 13 years with Morgan Guaranty Trust Co., was named chairman of the bank company almost five years ago. Moody will remain a consultant to the bank.