Insurance companies that charge different rates for men and women do not violate state human rights laws, a New York appeals court ruled yesterday. The opinion overturned a lower court ruling that different rates for life, health and liability coverage based on sex are discriminatory.

The ruling is the latest setback in the courts for women's rights organizations, which have been campaigning to end the industry's use of sex as a criterion for setting rates. But it comes at a time when a few states are moving legislatively and administratively to end the practice.

When sex is used as a rate factor, women tend to pay higher premiums for health insurance and receive lower benefits from retirement plans such as annuities. Men, on the other hand, tend to pay more to auto and life coverage.

The five-judge New York panel dismissed a $21 million class action brought by the National Organization for Women against Metropolitan Life Insurance Co. The court held that state law "has no application to {Metropolitan Life's} gender-based, risk classification, rate-making policies, which are expressly sanctioned by the insurance law."

William Toppeta, an attorney for Metropolitan Life, noted that the legislature had passed laws permitting distinctions since passage of the human rights law. "The legislature never intended for the insurance industry to be prohibited from considering gender," he said. "But gender is not the only factor we consider. We consider age, health history, habits, life style, occupation and so forth."

Emily Spitzer, an attorney for NOW Legal Defense and Education Fund, said the group was "very disappointed with the decision. The court held that the civil rights statute exempts insurance companies, so they will continue to discriminate." She said NOW is "seriously considering an appeal."

Similar cases are pending or have been decided in several other jurisdictions. The D.C. Superior Court recently dismissed NOW's suit against Mutual of Omaha, a ruling that NOW has appealed. The organization has also filed suit against State Farm in California. A similar case against another carrier is pending in Pennsylvania.

The long-standing controversy over the use of sex for setting insurance rates began in the early years of this decade when civil rights and feminist groups seized upon the issue as an example of an injustice against women. The insurance industry countered on economic grounds, claiming that a ban on sex as a criterion would not benefit women through lower premiums and could cause company insolvencies if all policies had to be changed to make men and women equal risks.

Efforts to pass federal legislation outlawing distinctions based on sex -- as well as race, color, religion and national origin -- distinctions in pensions and annuities, and in auto, health and life insurance foundered in 1983 but recently resurfaced.

Unisex provisions are contained in the Economic Equity Act of 1987, introduced in both the House and Senate on June 2. With a few exceptions, the bill would affect only individual insurance policies and pension contracts written in the future. (Group policies are already covered by federal statutes barring discrimination.) Moreover, the change would not take place for at least a year after the law is enacted.

Since insurance companies are regulated by the states rather than the federal government, most of the action has been at the state level. Six states -- Hawaii, North Carolina, Massachusetts, Michigan, Montana and Pennsylvania -- have adopted unisex automobile insurance rates. In addition, Montana has banned the use of sex as a risk category for all lines of insurance affecting health, life and retirement income. Now the insurance commission of Massachusetts proposes to make that state the first populous one to follow Montana's lead.

Montana's law was passed in 1983 and went into effect Oct. 1, 1985. A study of its impact on consumers was conducted by the state insurance department this past spring. The results were used by both sides in a renewed fight over unisex insurance. Legislators demanding repeal were finally thwarted when the governor vetoed the legislation.

From the survey of the 25 largest companies writing auto, health and life insurance in Montana came these findings:Auto premiums for drivers under age 25 showed the most dramatic change with an average increase of 49 percent for females and a decrease of 16 percent for males. The average rate increase for a family with a young female driver was 33 percent; for a family with a young male driver, a decrease of 8 percent. Life insurance premiums for women increased substantially while men had only a slight decrease in whole life premiums and a slight increase in term life insurance rates.

Health insurance premiums for women dropped an average of 13 to 16 percent and rose for a male an average of 22 to 28 percent, depending on age.

Only one company, Aetna, pulled out of Montana, while several new companies have come in.

Despite the apparently wild swings in premiums, the combined results tend to cancel each other out for most policyholders, according to Dave Drynan, who did the survey. Over a lifetime, a woman would benefit because what she gains in retirement benefits compensates for higher auto premiums. Men also derive a net benefit of lower auto rates over a lifetime.

Health and life insurance premiums are less of a factor for most people, because 80 to 90 percent of men and women who carry health and life insurance are covered by group policies that are already unisex.

The real losers, according to Drynan, are those mainly low-income working women who will pay more for auto insurance but who have no group life or health.

Massachusetts, which eliminated age and sex-based ratings for auto insurance in 1978, plans this year to introduce mileage as a factor for drivers for the first time, said commissioner Peter Hiam. This change, sought by NOW, would recover for women drivers much of what they lost to unisex rates since women drive less than men.

Hiam has also decreed the elimination of sex-based insurance for all other types of insurance, saying they serve no social purpose. "Why should a retired male school teacher be able to move to Florida {because he gets a bigger pension} while a retired female teacher stays home?" he asked.

The regulation, which is scheduled to go into effect in July 1988, will not require changes in existing policies.