The private mortgage insurance industry has changed in the past five years from a "low-risk, high-profit enterprise" to one plagued by underwriting losses that are likely to continue for years, according to a new analysis.

This transformation was brought about by a combination of unwise pricing and a sharp fall-off in appreciation rates for homes, resulting in an "enormous shift in the industry's exposure to risk since 1979," said the report, prepared by Moody's Investors Service.

"The industry will have to work through an unusually large portion of heretofore underpriced, higher-risk loans over a period which will not provide a sufficiently high dose of inflation to cure such inadequacies," the report said.

Moody's pointed particularly to adjustable-rate mortgages written between 1981 and 1984, which it termed "poorly underwritten and largely underpriced." It noted that the performance of these loans "will depend on interest rates over the next few years." But even if rates remain stable or decline, "the low level of home price appreciation" will continue to cause problems.

As a result, Moody's rated the claims-paying ability of the industry as an "A." Michael F. Molesky, Moody's mortgage industry specialist who prepared the report, noted that an A rating means upper-medium quality, investment grade but with the presence of elements that suggest possible future impairment.

"Weakened capital positions have reduced the ability of many major firms to weather very severe, prolonged economic declines without additional support, either through parent company relationships or outside agreements with reinsurance companies," the report said.

Steven P. Doehler, executive vice president of the Mortgage Insurance Companies of America, an industry trade group, said in a statement that the A rating was "a tribute to the industry" considering "the recent changes in mortgage finance, including deregulation and the worst recession since the 1930s.

"The mortgage insurance industry is a shining star compared with the performance of other property casualty insurers," the statement added.

"Moody's was very accurate to say that this is a very critical time for the industry," Doehler added in an interview, noting that the industry has aggressively added business in the past several years but much of it in the form of loans whose performance is not yet known.

Mortgage insurance, though often paid for by borrowers, is designed to protect lenders in the event of a default. While lenders' primary protection comes from their right to foreclose on and sell the collateral, such sales often do not cover all the costs. Mortgage insurance is used to bridge that gap by covering 20 or 25 percent of the loss.

Insurance of this type is normally required when the loan amount exceeds 80 percent of the property value, or when there is some unusual risk factor such as purchase of a property for investment rather than as a personal residence.

There was some $190 billion in private mortgage insurance in force at the end of 1984, according to MICA.

Like just about everything connected with real estate, private mortgage insurance grew rapidly during the 1970s. Dramatic increases in home prices, coupled with low, government-regulated mortgage interest rates cut losses, permitting insurers to obtain "high net returns on capital averaging between 15 and 20 percent," according to Moody's.

But by the end of the decade, high prices and rising interest rates brought the boom to an end, touching off fierce competition between mortgage insurance companies.

Because almost all private mortgage insurance is written on an annual renewal basis, "capturing a competitor's renewal business became a primary means" of maintaining premiums and market share as the origination of new loans declined.

Fearful of losing their "seasoned book" of insurance on older, fixed-rate loans, firms began insuring pools of mortgage-backed securities "at unreasonably low rates" and "insuring new instruments such as ARMs of unknown risks at rates no higher than standard fixed-rate mortgages," Moody's said.

The recession of 1981-82 brought the additional risk into sharp focus as loss ratios -- claims as a percentage of premiums -- shot upward, exceeding 100 percent in 1982 and '83.

"The stage was finally set for substantial revisions in the way many MI firms did business," Moody's said. Together, the companies and the Federal National Mortgage Association and the Federal Home Loan Mortgage Corp. -- which as giant buyers of mortgages in the secondary market "depend greatly on the continued viability of the . . . industry" -- have instituted changes in underwriting practices that "should have a beneficial impact on the industry over the long run," the report said.

The industry also has boosted prices in all product lines, Moody's said.

Nonetheless, the industry faces a long workout period, according to the report. "The risk potential of various ARM instruments will not be fully realized for several years and could be a threat to the industry if interest rates should rise sharply in the near future," it said.

Molesky noted that that most ARMs have been written in a climate of declining interest rates so the ability -- and willingness -- of borrowers to keep them up when rates rise is not known. He said he believes the performance of graduated payment mortgages, in which payments begin low and rise automatically at specified times, are a good proxy for ARMs in times of rising rates.

Most GPMs are structured so that payments increase 7 1/2 percent, which he said would be comparable to an ARM with a one-point cap.

Not everyone, however, regards the higher rates as justified.

Dennis J. Jacobe, director of reasearch for the U.S. League of Savings Institutions, a major thrift trade group, said that whereas the private mortgage insurance industry formerly used fixed-rate loans to set premiums for ARMS -- thus underpricing the insurance -- it is now going too far the other way in using graduated payment mortgages as a proxy for ARMS.

In his view, "there is no question" that GPMs are riskier than ARMs, and premium rates based on GPMs are unnecessarily high for ARMs.

"If they had differentiated between those kinds of products, then their portfolio would be viewed as less risky," he said.

Jacobe also said that for lenders the purchase of mortgage insurance "is more of a decision now." When premiums were much lower, "there was no question that it was a good purchase. Now with higher premiums and a weakened industry, I tell people in our industry they ought to seriously consider self-insurance."