The Interior Department yesterday put 540 million tons of federally owned coal on the market in defiance of a congressional resolution directing the department not to lease the coal for at least several months.

But what was billed as a constitutional confrontation had an anticlimax: 428 million tons drew no bids and will stay in the ground. The remainder, contained in five small tracts useful only as add-ons to existing adjacent mines, drew one bid per tract. Four bids came from one company and its subsidiaries.

The total amount offered for the government's coal in yesterday's sale, in the Fort Union region of North Dakota and Montana, was $911,800. The average bid of $110 per acre was $10 above the government's minimum per-acre price. The Treasury will also collect royalties of 12.5 percent on the sale price of the coal after it is mined.

Meanwhile, the controversy over Interior Secretary James G. Watt's accelerated coal-leasing program, particularly the Fort Union lease sale, intensified yesterday. Critics likened it to "the coal industry's surplus cheese giveaway."

Assistant Interior Secretary Garrey E. Carruthers dismissed the charge, stressing that Fort Union contains lignite coal, one of the lowest-valued types of coal. By next month Interior will reject any bid below the coal's fair-market value, he said.

"The only way this sale would have been a 'failure,' " Watt said, "would have been for the department to have yielded to pressures to scrap five years of costly study and nearly $1 million of research and called off the sale . . . . The coal leasing program is good for consumers, job seekers and national security. We are very pleased the market has had an opportunity to work."

Meanwhile, Democrats on the House Interior Committee caucused yesterday to decide how to respond to Watt's defiance of their resolution, which passed Aug. 3 on a 27-to-14 party line vote.

The committee invoked a provision of the Federal Land Management and Policy Act, which allows congressional committees to withdraw federal lands from use temporarily in "emergency" circumstances. The panel expressed concern over possible environmental damage from the strip mines, and also about the slack coal market.

Watt called the resolution unconstitutional.

The National Wildlife Federation and the Wilderness Society have filed suit seeking to have the committee's action upheld, since the statute at issue could be used to protect other federal lands.

"That statutory authority must remain valid," said an aide to House Interior Committee Chairman Morris K. Udall (D-Ariz.). "The committee acted under its authority. The secretary ignored the authority and proceeded anyway. The fundamental conflict is there even if nobody bids."

The issue of whether Watt has received a fair-market value for the government's coal, as required by law, flared last spring when the General Accounting Office reported that a 1982 sale of leases netted $100 million below their value. Watt said the report was inaccurate.

One result of yesterday's low turnout is that no new mines will be opened, since none of the large, new-production tracts drew bids.

Watt said that he expected only one bid on each of the five small tracts since they are useful only as "adjuncts to the existing operations" in the area.

Conservationists said the slack bidding sharply reduced environmental threats they had perceived from the Fort Union program. But they also said that they will still try to invalidate the lease sale.

"Eight thousand acres of coal mines and 114 tons of coal is nothing to sneeze at," said Norman Dean, the Wildlife Federation's director of public lands and energy issues.