The Carter administration, still roofing for ways to restrict U.S. oil imports, is exploring the idea of a licensing system under which the government would auction "rights" to sell overseas petroleum here.

The plan, a form of import quotas, is being considered as an alternative to imposition of an import fee of $4 to $5 a barrel if Congress fails to pass the President's proposed crude-oil excise tax in the next few weeks.

Both the excise tax and the import fee would discourage consumption by driving up the average price of a barrel of oil in this country. The licensing fee plan also would do this, an in addition give officials a way to restrict import volume directly.

Some administration officials believe the licensing system would be easier to put into place becuase it would not depend on congressional approval to the same extent as import fees. Many lawmakers oppose the fees.

Carter warned last week that if Congress did not act soon on his energy legislation, he would move on his own to limit oil imports by "administrative action." The energy bill is stalled in a House-Senate conference.

This morning, the Senate Finance Committee is slated to vote on a resolution opposing oil import fees. The Senate added to the energy bill an amendment revoking legal authority for the fees, but that is stalled along with the bill.

The plan to impose a flat import fee was opposed publicly yesterday by independent domestic oil refiners, who protested that such a measure would give a crucial edge to European competitors.

Robert E. Yancey, president of Ashland Oil Inc., told a hearing of the Senate Judiciary antitrust subcommittee that if import fees were levied, "the domestic refinery industry would be decimated.'

At the same hearing, a spokesman for the industry was quizzed about published reports that the administration offered oil producers billions of dollars in concessions if the producers help pass its crude-oil excise tax.

Charles DiBona, executive vice president of the American Petroleum Institute, first told the panel "there was no deal or the suggestion of one." Then he conceded, however; that trade-offs were "talked around."

The recommendation that Carter act on his own to impose oil import fees if Congress fails to pass the energy bill came originally from Treasury Secretary W. Michael Blumenthal.

G. William Miller, chairman of the Federal Reserve Board, also has supported such a step. Both have argued that failure to pass the energy legislation has been a major factor in the recent decline in the value of the dollar.

The plan to auction oil-import "rights" is being considered by administration policymakers as an alternative to the Blumenthal plan. One official described the two schemes as "the two most sensible" on the table.

Although the import fee plan is the simpler of the two, officials say the auction would have two advantages - it would bring in extra revenues to the Treasury and make it easier for the government to push up prices.

Under the plan, the administration would put a quota on the amount of foreign oil it wants to allow into the country and divide up that market in a system of licenses.

U.S. petroleum importers then would bid on the "rights" to bring in a specific amount of oil, with the license for each portion of the market going to the firm that bid the highest.

The system would make it easy for the government to push prices up. If demand for imported oil were running at, say, 7.5 million barrels a day, the administration could set the quota at 7 million barrels. The bidding - and prices - would climb.

C. Fred Bergsten, assistant secretary of the treasury for international affairs told the Senate Finance Committee yesterday an import fee similar to the crude-oil excise tax would save half a million barrels of oil a day.

However, officials stressed that Carter considers both the import fee and auction proposals second to the crude-oil excise tax he has proposed. "The President would rather have the energy bill," one strategist said.

The resolution in the Senate Finance Committee is being sponsored by Sen. Bob Dole (R-Kan.), with cosponsors. Dole said it may be modified slightly to deal with the domestic refinery issue.

At a Finance Committee hearing yesterday, Sen. Clifford P. Hansen (R-Wyo.), who supports the Dole resolution, asserted that if the United States indicates it is willing to pay higher oil prices, foreign producers will raise prices on their own.