After years of trying and failing, the Federal Aviation Administration yesterday published its final rule on the sticky question of how long airline pilots should be permitted to fly and how much rest time they must have between trips.

The rule appears to significantly narrow the gap between the FAA's fairly tough requirements for large airlines and those for the growing fleet of commuter airlines.

The rule was worked out largely in a "regulation by negotiation" process involving representatives of the pilot unions, the major airlines, the regional airlines, aviation users and the FAA. However, negotiators were unable to resolve some of the toughest issues and FAA Administrator Donald D. Engen had to impose solutions.

Many of issues addressed are labor-management questions that had tied up the regulatory process. The FAA said that the negotiation "acted to substantially narrow the number of differences between the parties . . . . "

The existing rule is 30 years old and was written in the days of propellers and piston engines. The new rule applies only to domestic flights and will take effect Oct. 1. Among other things, the rule:

* Requires scheduled commuter airlines to apply daily, weekly, monthly and annual flight-time limits, including a 34-hour flight time maximum in any seven-day period. Under existing rules, scheduled commuter crews can fly 10 hours a day every day.

* Permits unscheduled commuter airline pilots to fly a maximum of 500 hours in any calendar quarter but only 1,400 hours annually. The purpose is to permit unscheduled airlines with very seasonal operations to work their crews hard during the season, but limit their flying the rest of the year.

* Continues a requirement that regional airlines flying planes with 31 or more seats must impose the same flight-time standards as major airlines. Many regional airlines fly mixed fleets with some planes of 30 or fewer seats and some with 31 or more, and they sought identical treatment for both classes. National Transportation Safety Board Chairman Jim Burnett vigorously opposed that concept.

* Requires big airlines to guarantee pilots a minimum of eight hours' rest between shifts. Under the existing rule, there is no minimum requirement. The airlines had wanted a 7 1/2-hour minimum; the pilots, nine hours. Existing limits of 1,000 flight hours a year, 100 flight hours a month and 30 flight hours in any seven consecutive days remain.

* Permits some shortening of the rest period if extra rest is provided the next day. This gives airlines an opportunity to adjust to the real-time problems of delayed flights.

Alan R. Stephen, vice president for operations of the Regional Airline Association, said the regulation will "put substantially increased record-keeping requirements" on commuter airlines because the rules outline weekly and monthly limits as well as daily limits.

The FAA said it moved ahead on the rule without considering "recent biomedical research or factors affecting pilot fatigue" because of the "overriding importance of clarifying and simplying the current rules."

The FAA noted that the National Aeronautics and Space Administration is studying pilot fatigue and has scheduled a report for late in 1986. "If the report establishes a quantifiable relationship between fatigue and job performance and identifies specific criteria which support an amendment," the FAA said it would consider additional rulemaking.