President Reagan, citing foreign policy concerns, has ordered the Justice Department to call off a year-long federal grand jury investigation into possible criminal antitrust violations by airlines flying between the United States and Britain, the Justice Department announced yesterday.

The unusual presidential decision was taken over the objections of the Justice Department, where lawyers said they could not remember when a president had last shut off a grand jury criminal investigation.

Reagan acted less than a month after the British government escalated a long-standing dispute with Washington over the application of U.S. antitrust laws to foreign airlines by refusing to allow new low transatlantic fares on the grounds they could form the basis for future antitrust suits.

The president's action, taken late last week, was seen as going a long way toward meeting the antitrust complaints of the British government. It was unclear, however, if the government of Prime Minister Margaret Thatcher would allow the low transatlantic fares to go into effect as a result of the presidential action.

Justice Department spokesman Mark Sheehan denied any connection between the presidential decision and any British move to allow the lower fares, which had been requested by major airlines flying the Atlantic route. The winter fares were to start Nov. 1 and continue through March.

Although U.S. and British government officials have been holding talks on both the fare and antitrust issues, Sheehan said the two never have been discussed in one set of negotiations. "There was no quid pro quo," he insisted.

Sheehan said Reagan's decision was "based on foreign policy reasons," which he declined to specify. "You can't expect the president to explain foreign policy decisions. It is enough for him to consider the factors and decide."

The Justice Department, taking a strictly legal view of the case, wanted to continue the grand jury investigation, Sheehan said, but the State Department wanted it dropped. As a result, the dispute was thrown to the president, who sided with State.

"Justice obviously was not willing to close the investigation of its own volition," said Sheehan.

"If the Justice Department had thought the investigation should be closed on purely legal grounds," he added, "the question would not have gone to the president in the first place. There was not complete agreement between the agencies as to which considerations were paramount -- legal or foreign relations," said Sheehan.

But Sheehan said closing the grand jury probe was an "appropriate exercise" of presidential power based on Reagan's constitutional positions as chief law-enforcement officer and director of the nation's foreign policy. "He determined that closing the investigation was in the national interest," Sheehan added.

Reagan made his decision after conferring with top officials from the White House and the State and Justice departments, Sheehan reported. Attorney General William French Smith was recused from the case, the Justice spokesman said, and the highest department official involved was Deputy Attorney General Carol Dinkins. The State Department had no comment on the presidential decision.

While the international dispute over extraterritoriality centered on antitrust issues, the British government long has battled American efforts to extend its laws past its borders in areas as diverse as export controls and banking.

British officials fought the U.S. court's jurisdiction over their airlines in the civil antitrust suit brought by liquidators of Sir Freddie Laker's bankrupt airline, but were overruled by the House of Lords, acting as the United Kingdom's highest court. They also tried to stop the grand jury probe, which covered much the same ground.

The civil suit and the grand jury probe, moreover, could affect the Thatcher government plan to sell British Airways to private owners. British Airways' sales value would be diminished by a civil triple-damage antitrust suit and a criminal grand jury investigation hanging over its head.

Besides British Airways, defendants in the 1982 civil antitrust suit, filed in U.S. District Court in Washington, were Pan American World Airways, Trans World Airlines, Lufthansa Airlines, Swissair and British Caledonian Airlines, as well as McDonnell Douglas Corp., an aircraft manufacturer, and McDonnell Douglas Finance Corp.

The suit accused the airlines of lowering fares to drive Laker out of business and getting together to squash a rescue attempt being put together with the help of McDonnell Douglas. Laker was forced into bankruptcy in 1982 after other airlines lowered their fares to match him.