President Carter's planned new $10.3 billion oil-import fee, which is expected to raise gasoline prices across the nation by 10 cents a gallon beginning May 15, may be in trouble in Congress.

At an initial hearing on the measure, members of the House Ways and Means trade subcommittee indicated there would be clear majorities, both in that group and in the full committee, for a House-Senate resolution rolling back Carter's action.

Opposition to the import-fee was so heavy that Ways and Means Chairman Al Ullman (D-Ore.), who previously had supported the president's position, publicly backed away and advised the administration to pursue some other course.

Ullman told Energy Secretay rChalres W. Duncan, "I have grave doubts that you will be able to survive" current pressures against the import fee. He said the opposition "may even be strong enough to override a veto."

Although Carter already has put the import fee into effect, Congress has the right under a provision in the recently enacted crude-oil tax bill to overturn his action by passing a simple joint House-Senate resolution.

The president could veto any such measure, but the lawmakers could then override him by two-thirds votes in both House and Senate. Carter would thus have to muster 34 votes in the Senate or 146 in the House to win.

The resolution of disapproval now before the subcommittee has more than 100 cosponsors in the House and 26 in the Senate, with still others supporting related bills. Only three members of the trade panel yesterday backed Carter on the fee.

The president imposed the oil-import fee March 15 as part of his new anti-inflation program, largely to provide a caution of additional revenues to help guarantee that the fiscal 1981 budget would be balanced.

The House and Senate Budget committees have included the extra revenues from the import fee in their own budget-balancing proposals, with directions that any surplus he used toward a 1981 tax cut.

Congressional leaders say that without the added revenues from the import fee, there would be no way to enact a tax cut this year without incurring a large deficit. Subcommittee member James R. Jones (D-Okla.) argued that point yesterday.

However, the import fee was opposed vigorously by a coalition of Republicans, oil-state congressmen and New Englanders, who charged the levy was only a revenue-raising ploy and could pave the way for higher heating-oil prices.

The issue is a sensitive one. Carter has established a complex collection system that he has promised will require refiners to pass the fee on to consumers only through higher gasoline prices, not by raising heating-oil charges.

But critics, including some in the oil industry, say the system is not ironclad and contend the refiners could pass the fee on in the form of higher heating-oil prices, too. Experts differ on the question.

The administration ran into further difficulty yesterday as a House Government Operations subcommittee voted to subpoena Energy Department papers supporting critics on that issue. The White House has refused to turn the papers over.

The action sets the stage for a possible contemp-of-Congress citation against Duncan if he fails to give the lawmakers those documents by Tuesday. Duncan contends the papers are private internal memos.

The Ways and Means subcommittee is expected to wait about two weeks before drafting the resolution to roll back the import fee. Panel chairman Charles A. Vanik (D-Ohio) supports the levy, but says he won't try to block a rollback.

Yesterday's session was filled with strong, often emotion-laden criticism of the import fee, including an exchange in which Rep. Edward J. Markey (D-Mass.) told Duncan: "You ought to be called the Secretary of Taxation."

At the same time, however, Rep. Sam Gibbons (D-Fla.) praised the administration "for having enough guts to do what is correct." He called the issue "a psychological problem, rather than an economic one."