An internal Education Department report has recommended major personnel cuts in programs that primarily benefit minority and disadvantaged children -- raising new charges that the Reagan administration plans to use layoffs and budget cuts to eliminate programs it has been unable to get Congress to end.

"These recommendations are an attempt to circumvent the will of Congress and the people in terms of their commitment to the disadvantaged, poor and minorities," said Warlene Gary, an official at the National Education Association, one of the nation's largest teachers unions.

"This administration has been unable to kill the department from the outside, so now it is attempting to kill it from inside . . . by weakening programs," Gary said.

Anne Graham, a spokesman for Education Secretary Terrel H. Bell, said the recommendations are "only that--recommendations -- and they should be viewed in that context."

"The secretary encourages all of his staff to make recommendations," Graham explained. "He has not reviewed it this report to date." She did not say when Bell would read the report or act on its recommendations.

The recommendations were made by a special task force of senior agency managers selected by Bell in May. Their recommendations, according to an analysis prepared by the NEA, call for the elimination of 597 full-time-equivalent positions during fiscal year 1983. The agency was budgeted 6,055 positions during fiscal 1982.

The bulk of the cuts would be made in the Office of Elementary and Secondary Education and would be aimed at the department's largest federal program -- Title I, a $3 billion program that provides states with money to help disadvantaged children improve their math and reading skills. The Reagan administration recommended cutting the program by $1 billion for the current school year, but Congress reduced the cut to $100 million. The task force recommended eliminating 89 of the office's 319 full-time-equivalent positions.

The task force report said those personnel cuts were justified because "direct monitoring" of the program has become a "non-priority activity." NEA contends that if the agency stops monitoring Title I funds, states will misuse the funds. A study by the Children's Defense Fund showed that disadvantaged children often were ignored in the early 1970s when Title I money was not closely monitored by the federal government, the NEA said.

The second major change would be a "shift of emphasis" in the Office for Civil Rights. The task force recommended that the office's resources be "shifted from a compliance effort to technical assistance" and that 41 of the office's 1,011 positions be eliminated.

"The policy implication of this is clear: the Reagan administration will simply not conduct the thorough on-site compliance reviews that the courts have concluded are vital to effective implementation of the nation's civil rights laws," NEA said.

The task force did recommend adding 69 more positions to the inspector general's office, bringing the total to 332 positions by fiscal 1984. It also recommended creating a 12-person "policy issues staff" and adding three lawyers to the department's legal staff to focus on eliminating agency regulations, NEA said.

Bell had asked the task force for recommendations on how he could "systematically redirect" the agency staff toward the administration's "highest priorities" while continuing to reduce the size of the agency and eliminate "unproductive or inappropriate" activities.

Tom Stack, who directed the task force, declined comment, but the task force final report shows that highest priority was given to programs that were both well-regarded by the Reagan administration and mandated by law. Next came programs that were strictly administration priorities. Low-priority programs included any items that were "not priorities of this administration . . . but which must be maintained because they are directed either by law, executive order, regulations or court order." At the bottom of the list were programs that were neither administration priorities nor mandated by law.