The crash last week of a DC-10 in Chicago is expected to become the most expensive accident in U.S. airline history and could total $100 million in payments to the families of victims, according to the leading insurer for American Airlines.

The estimate, offered by John Brennan, president of United States Aviation Insurance Group, is based on other recent airline cases in which awards have averaged several hundred thousand dollars for each person killed. It does not include the value of the aircraft, which is valued today at roughly $35 million.

Because the Chicago crash appears to have been the result of mechanical failure rather than human error, the resulting lawsuits are likely to rest on product liability claims, according to insurance experts interviewed yesterday. It is expected that both McDonnell Douglas of St. Louis, maker of the DC-10, and the federal government, which has responsibility for aircraft safety, will be sued along with American Airlines - and that McDonnell Douglas eventually will carry the brunt of the cost.

The accident, which took the lives of 273 people, comes at a time the rates for aviation insurance are at their lowest ever. But insurance experts were doubtful that what happened in Chicago will cause rates to rise.

"People have been projecting the rates would harden for some time now," said Brennan. "But despite several major accidents in the past few years, they've continued to decline. The rates have never been lower."

The reason for the steady slide has been a surplus of insurers, which has in turn led to fierce competition among insurers for airline business.

Insurance firms have been drawn to the aviation field by continuing improvements in the safety record of the airlines. Worldwide figures for 1978, published by the International Air Transport Assoication, give a fatal accident rate for jets of one in every one and a half million flying hours. And the numbers for the U.S. airline business are better than that.

(Paranthetically, the trend in general aviation in the U.S. has been just the reverse. Government figures show a rise both in accidents and fatalities in small planes last year over the year before.)

Even with the airlines' improving safety record, the insurance industry had expected - and had hoped for - rates to rise. The number of people traveling by air has been up, thus increasing the chance that a major accident will mean a long casualty list. and those few major accidents that have occurred have been costly ones.

From the early days of commercial aviation 50 years ago, airline insurance has been underwritten primarily by consortiums. Two major pools exist in the U.S. - the U.S. Aviation Insurance Group and the Associated Aviation Underwriters, both in New York. Insurance pools also exist in most parts of the developed world, sometimes providing the only local underwriting and sometimes facing competition from single insurers.

Many airline fleets receive their insurance ratings from London-based insurance groups which also provide much of the reinsurance - that is, insuring of insurance policies - which has become popular in the field. Major airlines customarily carry accident coverage totaling $500 million.

Just how much the family of a crash victim receives is the result of frequently protracted litigation. In cases involving international flights, the airlines' maximum liability has been set by several multilateral treaties - the Warsaw Convention. Montreal Agreement and Hague Protocol.

In contrast, cases involving domestic fights are not subject to any maximum, except in West Virginia and Kansas. Many other states once had ceilings but have abandoned them.

Nonetheless, there can be significant differences among states on the kinds of considerations that will be allowed in computing damages in a crash case. In general, the age and occupation of a crash victim is given the most weight to determine what earnings have been lost to those who might have depended on the victim.

Several states go further and allow recovery for the mental anguish of those surviving. California allows for "loss of care, love and affection." In other instances, recovery for loss of parental care and guidance is permited by state courts.