The Senate Energy Committee added a $100 million-a-year sweetener for coastal states as it reported out a bill to overhaul procedures for awarding oil drilling rights on the nation's outer continental shelf.

Meanwhile, the House Ways and Means Committee indicated in a series of votes that it will approve, but in softened form, the tax and rebate part of president Carter's energy package intended to convert industry and utilities from burning oil or gas to coal.

The purpose of the outer continental shelf bill is to provide more competition and otherwise see to it that the government gets its money's worth in selling drilling rights for oil and gas in submerged lands owned by the United States up to 200 miles off its shores.

Under present procedures leases are awarded to the highest cash bidder. To spread the action among smaller operators, the bill provides that at least one-third of the leases must be awarded under alternative bidding methods, such as agreeing to pay the government a specified royalty on any oil or gas discovered.

Sen. J. Bennett Johnston (D-La.), chief committee spokesman for the oil industry, tried to knock out this provision, and was defeated 10 to 8. But Johnston won two other important victories yesterday.

First, by a vote of 11 to 7, the committee struck from the bill provisions empowering the government to do exploratory drilling before selling leases to find out which tracts of underwater land contain oil and what the leases might be worth.

The Secretary of the Interior has power to contract for such drilling. But the drillers presumably would be oil companies, and the government would have to rely on their information. The provision giving the government power to drill helped kill the bill in the House last year.

Then Johnston, allied with Chairman Henry M. Jackson (D-Wash.), got through an amendment to pay coastal states rolyties for oil produced off their shores. Sponsors justify the payment as compensating for environment an dother impacts of off-shore.

The bill would authorize 100 million a year for the next two years for these payments. The largest single amount would go to Johnston's state of Louisiana. The government makes similar payments to states for coal mined on federal lands.

The bill would also give coastal states a large say in offshore drilling policy, create a new oil-spil liability system under which lessees where the spills occur would pay cleanup costs and the first $35 million in damages and require leaseholders to file exploration and well as development and production plans so federal and state governments would know what they plan to do at each state. Lessees would be required to display "due diligence" in drilling and not sit on their leases waiting for prices to go up.

The coal conversion plan, on which Ways and Means resumes action today, is expected to be the big oil saver in Carter's energy package. It would raise $90 billion in taxes by 1985 and give $50 billion of the back in rebates to industry and utilities that convert from oil or gas to coal, which the nation has in abundance.

The administration believers that by 1985 this conversion tax program would mean savings of 3.3 million barrels of oil a day, or its equivalent in natural gas. The administration's savings estimate for its entire energy package is only 4.5 million barrels.

The committee worked yesterday on a staff proposal which would change the administration's one-rate tax to a two-ter tax. Industries that could convert to coal without difficulty would be axed at a higher rate. Those that could not convert or could do so only with difficulty would be taxed at a lower rate to encourage conservation of oil or gas. The staff proposal inclued lower tax rates than the administration's.

By a vote of 18 to 17 the committee rejected an amendment that would have cut the program by more than half by limiting the tax to boilers.

Then, by a vote of 22 to 7 it exempted processing fuels such as used in the textile, glass and auto industries where conversion to coal would have an adverse effect on the product.

Rep. Abner J. Mikva (D-Ill.) succeeded, 14 to 10, in raising the tax rates close to the administration figures in later years. Mikva estimated that the day's voting had knocked about one-third off the administration's hoped-for savings.