The United States and Soviet Union are discussing resumption of airline flights between the two countries as part of complicated negotiations aimed at protecting airliners from straying into Soviet territory.

The talks, which include Japan, were spurred by the disaster involving Korean Air Lines Flight 007 on Sept. 1, 1983.

Unaccountably 310 miles off course, the plane was shot down by Soviet fighters over Soviet territory, and the 269 people aboard were killed. The incident terminated the possibility that tension between the superpowers might have eased before the U.S. election campaign.

U.S. government sources said a new security agreement could provide direct communication between U.S. or Japanese civil air-traffic controllers and Soviet authorities if a plane wanders off course on heavily traveled North Pacific routes between Anchorage and Tokyo.

"We're basically talking about procedures to be followed in an abnormal situation," one source said.

Various technical possibilities are under discussion, ranging from a type of hot line to electronic navigation checkpoints provided by the Soviets for airliners.

Several tripartite meetings were held between Feb. 26 and March 3. A second series is pending. Sources described the negotiations as sensitive and said they include technical and political issues.

Another source said "there is direct linkage" between achieving an agreement and permitting Aeroflot, the Soviet airline, to resume flights to the United States. "Aeroflot has wanted to get back in from the day we threw them out," the source said.

President Reagan suspended all U.S.-bound Soviet flights Dec. 29, 1981, as partial retaliation for what he described as Soviet "repression in Poland."

Pan American World Airways, the major U.S. international carrier, flew between New York and Moscow for 10 years but abandoned that service in October 1978 for economic reasons. The Soviet Union hampered Pan Am by requiring that its citizens fly Aeroflot and by forbidding Pan Am to sell tickets in the Soviet Union.

Pan Am is interested in returning to Moscow, however. Its chairman, C. Edward Acker, said in an interview that he hopes to begin service there by June 1986. A company spokesman said Pan Am and Soviet officials have held discussions, and a U.S. government official said, "Aeroflot wants Pan Am to lobby for them."

Pan Am has been expanding flights to Eastern Europe from its base in Frankfurt and last Sunday resumed daily service to Warsaw after a break of several years.

But the airline cannot be given landing rights in Moscow while the Soviets remain barred from the United States, and improved communication and security for planes on North Pacific routes such as that of Flight 007 are considered essential first, U.S. sources said.

At one point, that route comes within 11 miles of Soviet territory. Flights on North Pacific routes maintain radio contact with U.S. or Japanese controllers but are beyond range of civilian radar surveillance for 1,151 of the 3,299 statute miles between Anchorage and Tokyo. Thus, there is no external check on the accuracy of an airplane's on-board navigation equipment. Misprogramming of computerized navigation equipment and inattention by the flight crew are the generally accepted explanations for Flight 007's excursion, but the Soviets and some Western commentators have suggested that the plane was on a spy mission for the United States.

The State Department has strongly denied that charge.

Since Flight 007, the Federal Aviation Administration and the Air Force have beefed up radar surveillance in that area, particularly in the first few hundred miles beyond Anchorage, and have improved coordination of civil and military data. Nonetheless, wide gaps remain in ground-based civilian radar coverage.

In addition to improving security, the U.S. sources said, Aeroflot's return would be contingent on:

* Agreement on claims of survivors and other parties arising from the KAL incident, a matter before U.S. District Court here. More equitable balance between benefits for Pan Am and Aeroflot than existed in prior agreements. Pan Am, which is redefining itself as a European-oriented carrier, would not be looking to Moscow if it did not think it could make money on the route, Pan Am officials said.

Acker said Pan Am probably would use a twin-engine, standard-body Boeing 737 between Frankfurt and Moscow if flights were resumed. Pan Am uses the much larger Boeing 747 jumbo jet for many transatlantic flights.

U.S. government sources said reopening direct airline flights between the two nations would be more advantageous for the Soviets than for the United States. One source said Aeroflot flights to the United States provide more convenient travel for Soviet officials stationed in the United States than the multiple change of flights and airlines now necessary.

Loss of Aeroflot flights cost the Soviets $20 million in hard U.S. currency annually, and the Soviets occasionally used Aeroflot for intelligence purposes, the source added.

The source cited two 1981 Aeroflot flights -- one over Pease Air Force Base, N.H., and the other over General Dynamics' Trident submarine works at Groton, Conn.

Both planes were bound for Washington Dulles International Airport and were to fly over the Atlantic Ocean but ignored instructions from U.S. air controllers to return to their proper routes, the source said.