The Interior Department has accused 10 coal mining concerns of illegal drilling in federally owned coal reserves in eastern Wyoming, charges that could result in heavy fines for the firms and jeopardize Interior's record-breaking coal leasing program in the area.

Interior's Bureau of Land Management issued notices of trespass against 10 concerns, including Kaneb Services Inc. of Houston and Northwestern Mutual Life Insurance Co. of Milwaukee, Pat Korp, speaking for the BLM in Wyoming, said yesterday.

Kaneb and Northwestern Mutual were partners in an aborted $1.79 billion project to develop synthetic fuels using coal from eastern Wyoming's mineral-rich Powder River Basin.

Korp said the companies drilled without permits into federally owned coal reserves to obtain data on the quality and quantity of coal in the area. She said the department's inspector general is investigating whether this gave the firms an unfair advantage last April in bidding on more than 1 billion tons of federally owned coal--the largest offering of government coal reserves.

Three of the firms cited for trespassing were awarded leases by Interior last year. These were Northwestern Mutual and a joint-venture partner, Texas Energy Services Inc. of Gillette, Wyo., which bid $22.3 million for their lease, and Neil Butte Co. of Gillette, which bid $1.6 million for another tract.

The trespassing charges, issued under the federal coal leasing law, carry a maximum fine of $1,000 a day. Korp said the drilling apparently began in 1977 and that penalties could run "into the millions."

A spokesman for Northwestern Mutual said the company had not been informed of the charges. Officials of other firms cited by BLM could not be reached yesterday.

The trespassing notices are the latest in a series of problems with Interior's coal leasing effort in the area. The General Accounting Office and the House Appropriations subcommittee on interior affairs are investigating charges that Interior officials improperly leaked confidential data to the industry before bids were submitted last April. The Powder River lease was denounced at congressional hearings as a "giveaway" because it was awarded in a glutted coal market and drew little competition.