The Organization of Petroleum Exporting Countries today rejected a Libyan request for "joint action" against the United States in retaliation for the Reagan administration's decision to call on all American oil company employes and their dependents to leave the North African nation.

OPEC President Mana Saeed Otaiba of the United Arab Emirates said the 13-nation conference had listened to the complaint of the Libyan delegation regarding the U.S. action but decided that the issue was of a "political nature" and should be left to individual governments to handle.

Conference sources said the Libyans submitted a two-page draft declaration condemning the United States and calling for an OPEC oil embargo as well as the blacklisting of any American companies leaving Libya.

Bowing to the glut on the world market, OPEC also announced cuts of between 20 and 70 cents in the official price of a 42-gallon barrel of mainly heavy fuel oils used to heat homes. Oil experts here estimated that the reduction, while a welcome continuation in the downward trend of oil prices, would change the cost of a gallon of gasoline by only a fraction of a cent.

Only Iran appeared to back the Libyan call for reprisals against the United States. An Iranian official told reporters that the delegation proposed that U.S. companies pulling out of Libya should be penalized and not sold oil.

But OPEC moderates led by the Arab members, already irritated over a Libyan decision to oppose the admission of Tunisia into the separate organization of Arab oil producers, refused to cooperate.

"We always thought of this organization as an economical gathering, and we cannot deal with political matters here," said the Saudi oil minister, Sheik Ahmed Zaki Yamani. "Probably this is why we didn't take any decision."

Libyan Oil Minister Abdessalam Zaggar said yesterday that he was hoping to obtain assistance from other OPEC members to overcome problems his government anticipated in trying to run the oil fields after the departure of the 1,500 American technicians and their dependents.

There was no indication tonight that Zaggar obtained any offers of aid from other OPEC members.

Most of today's session was reportedly spent debating how much prices should be cut in light of the difficulty many OPEC members are having selling their oil.

The biggest surprise at the conference was the reversal of Iran, which has long been regarded as a hawk in pricing matters. Instead the Iranian delegation reportedly said that it was willing to cut the price of its heavy fuel crude by as much as $1 a barrel in order to restore its production to at least 2 million barrels a day by next year.

The organization finally agreed to a 70-cent reduction on Iranian and similar quality Kuwaiti oil, while Saudi Arabia agreed to a cut of 60 cents in its comparable crude.

Meanwhile, Libya and Algeria, whose oil price is at the top of the North African producers, stuck officially to the $3 to $4 premium on their light, or low sulfur, oil, over the benchmark oil price of $34 a barrel. But Yamani indicated he expected them to come down by 50 cents to $37 a barrel and said they might find it "a little bit difficult" even to get that amount.

Yamani said that oil prices were on a downward trend because of the oil glut and that the reductions announced today would affect mainly heavier fuel oils used in home heating and factories.

"Instead of going up in prices as usual we are now coming down," he said, adding that if fuel oil prices continue to be depressed further cuts might be ahead.

But he and Otaiba said they thought Western demand for oil would begin to pick up by the middle of next year because stocks were being rapidly depleted and there was hope for some recovery in the current economic recession.

Yamani said he was "extremely happy" with the results of the OPEC meeting, which served largely to confirm the Saudi thesis that the top prices charged by OPEC members are still too high and that the spread between the Saudi benchmark price and additional premiums charged for top quality oil should be narrowed.

Only seven weeks ago, OPEC decided at an emergency meeting in Geneva to close this price gap from $9 to $4. Today it effectively agreed to close it yet another dollar, while granting even larger discounts from the $34 benchmark price for medium and heavy crudes.

But OPEC's actions are simply reactions to market conditions that have made it difficult for not only the hawks but also the moderately priced producers to find customers for their oil.

Total OPEC production has dropped nearly a third from two years ago to around 20 million barrels a day in September, although an OPEC official said today he expected production to average 22.5 million barrels during the last quarter of this year.

Oil experts estimated that about 8 million barrels, or 40 percent, of OPEC's total production would be affected by today's price cuts, which go into effect Jan. 1. The probable cut in the price of a gallon of oil would be between two- and three-tenths of a cent, not enough to make a difference to consumers. But Yamani insisted the effect would be felt even among non-OPEC producers and to some extent on all prices across the board because of the general downward trend.

Regarding OPEC's efforts to devise a long-term pricing strategy, the Saudi oil minister indicated the organization was still far from agreeing on a formula.

He said countries with limited reserves like Libya, Iran and Algeria had a fundamentally different view from those producers with large deposits like Saudi Arabia. More studies were required to find "a flexible pricing formula" acceptable to all, he said.

The Saudi oil minister is head of a special OPEC committee that has proposed a steady graduated increase in oil prices based on world inflation rates, changes in the value of major Western currencies and real growth rates of Western economies.

The oil glut has shown again that the price decisions of OPEC producers continue to be determined primarily by market conditions, making a long-term strategy extremely difficult to follow.