The Carter administration has invoked executive privilege in refusing to explain how it came up with a scary set of unemployment projections it used to help stop last winter's coal strike.

The executive privilege claim, usually reserved for the most sensitive presidential documents, was made to General Accounting Office investigators studying federal actions during the strike.

Unemployment estimates, ranging as high as 3.5 million, were critical in the administration's efforts to win a court order forcing striking United Mine Workers members back to their jobs.

In seeking a back-to-work order under the Taft - Hartley law, the White House had to demonstrate that a national emergency existed.

The information was withheld by the White House's Council of Economic Advisers, on advice from the Department of Justice, when the GAO asked for background on the economic-impact calculations of the strike.

An unsigned Justice memorandum, dated Aug. 31, held that, since the CEA had prepared the data for President Carter and for use during court proceedings to halt the strike, executive privilege was applicable.

Even without that data, however, the GAO concluded that the administration used "questionable" statistical methods that "did not present a fair assessment of the situation to the public."

The GAO report, issued last month, said weather problems had a greater impact on general industrial unemployment than the miners' strike.

Citing Bureau of Labor Statistics findings, the GAO said that no more than 25,500 industrial workers were laid off directly because of the strike.

The executive-privilege issue and the Department of Energy's activities to deal with the strike emergency will be discussed next week at a House energy and power subcommittee hearing.

Rep. John D. Dingell (D-Mich.), subcommittee chairman, has asked officials of the Energy Department, the GAO and the CEA to appear before his panel next Friday.

Dingell said he considered the Justice Department opinion "an arrogant and presumptuous statement, reminiscent of those made by the Justice Department during the term of former Attorney General [John] Mitchell."

Executive-privilege claims, protecting certain presidential papers from disclosure, were made by the Nixon administration in connection with the Watergate case.

The Carter White House, publicly supportive of "open government," has been reluctant to apply executive-privilege claims.

Dingell indicated that he felt the GAO had full authority to look at the background data, since it was acting as an arm of his subcommittee, which had requested the investigation of DOE actions last winter.

The GAO found generally that the department has done inadequate planning and is not prepared to deal quickly and smoothly with national energy emergencies.

In response to the GAO and a similarly critical study by DOE's inspector general last summer, agency officials said that planning procedures have been intensified and improved.