Congress' two tax-writing committees yesterday picked up where the federal courts left off and voted to kill President Carter's new oil-import fee, possibly wounding House and Senate budget-balancing efforts as well.

First, the Senate Finance Committee voted, 14 to 4, to repeal the fee, which would have raised gasoline prices 10 cents a gallon today had it not been blocked Tuesday in a U.S. court.

A few hours later, a House Ways and Means subcommittee approved a similar resolution of disapproval, 17 to 4, ignoring Treasury Secretary G. William Miller, who had asked it to delay its vote pending appeal of the court ruling.

Both House and Senate are now thought likely to vote overwhelmingly to repeal the import fee just after Congress' Memorial Day recess later this month. Repeal would seriously jeopardize Congress' budget-balancing efforts.

Both houses have counted, in their estimate of revenue for the next fiscal year, the $10.3 billion that the import fee was expected to raise. The House had earmarked the money for a posssible tax cut.

Meanwhile, U.S. District Court Judge Aubrey E. Robinson Jr., who ruled Tuesday that the import fee was illegal, yesterday agreed to allow the government to maintain record-keeping requirements for oil companies pending appeal.

His decision means the basic administrative apparatus for computing and collecting the fee will remain intact so the fee can be reinstated quickly if the government wins at the appellate level.

However, no money or fees will be collected from the oil companies, and the levy cannot be passed on to consumers, as Carter had planned for this morning. The appeal is expected to take several weeks.

The action by the two tax-writing panels came under a special provision of the recently enacted crude-oil tax that allows Congress to block by simple resolution any new restrictions the president imposes on oil imports.

Carter already has served notice that he will veto any such measure, but congressional leaders appeared confident yesterday they will be able to muster the two-thirds majorities of both houses needed to override him.

Rep. Charles A. Vanik (D-Ohio), chairman of the Ways and Means subcommittee, described the rollback resolution as "veto-proof . . . I think, probably in both houses," a view shared by full committee chairman Al Ullman (D-Ore.).

The full Ways and Means Committee is expected to approve the rollback resolution this morning, with plans to send it to the House floor in early June. The Senate will consider the proposal after the House votes.

The Vanik panel had been prepared to approve the measure Tuesday, but agreed to put off action until yesterday in deference to Miller, who had asked permission to testify. The secretary had declined to appear previously. a

Yesterday, Miller asked the panel to defer action further pending resolution of the appeal in court, and defended the import fee as a necessary oil-conservation measure, not just a budget-balancing ploy.

Miller at one point interrupted his testimony to read the committee a wire-service story announcing that Saudi Arabia had just increased its crude-oil prices by $2 a barrel.

He called the new price increase "another dramatic indication" that the United States had better begin taking more steps to cut oil imports. He also alluded, less directly, to the problems that repeal would cause for the budget.

However, like Energy Secretary Charles Duncan before him, Miller ran into a buzzsaw of criticism from panel members, who dismissed the import levy as primarily a budget-balancing gimmick that actually would do little to cut consumption.

Rep. Tom Downey (D-N.Y.), a vocal opponent of the new fee, told Miller if the administration had proposed spending $10.3 billion to conserve 100,000 barrels of oil a day, as the fee would, "you'd have been laughed off the Hill."

And several lawmakers told the secretary that the administration should turn to alternative proposals for cutting gasoline consumption, such as a 50-cent-a-gallon gasoline tax or faster decontrol of oil prices.

The administration has considered most of these options before, but has backed away from proposing them, mostly because congressional leaders have predicted they would be defeated.

That difficulty was noted in a prevote appeal by Vanik, who complained that voters want to conserve energy, but "they don't want higher prices, they don't want higher taxes -- that's unachievable."

Rep. Sam Gibbons (D-Fla.), another panel member opposed to repealing the import fee, also challenged critics to come up with "a positive program of their own," but was defeated in a bid to delay the vote.

Subcommittee members voting against repeal included Gibbons, Vanik, Dan Rostenkowski (D-Ill.) and James R. Jones (D-Okla.). Joseph L. Fisher (D-Va.), the only local member of the panel, voted to roll back the levy.

In the Senate Finance Committee, the four voting against a rollback were Lloyd Bentsen (D-Tex.), Abraham Ribicoff (D-Conn.), Spark Matsunaga (D-Hawaii) and Robert Packwood (R-Ore.).

In court papers filed yesterday, government attorneys asked for a quick decision from the appeals court and proposed that arguments be scheduled for the week of May 26, but opponents are expected to challenge that.