The Labor Department is suffering serious management problems and a backlog of work, brought on by budget cuts, the shock of nine major layoffs at the department in four years, a high turnover rate among its political appointees and poor planning, a new General Accounting Office report to Congress contends.

Previous GAO reports have discussed the disruptive effects of layoffs that hit a number of federal agencies during the first three years of the Reagan presidency. But the GAO says the impact of the layoffs was especially damaging at Labor, which went through nine different cutbacks that affected 20 percent of its staff.

The congressional watchdog agency contends in its regular management review of the department that some of the problems stem from the tenures of past secretaries of Labor who spent more time dealing with Congress and constitutent groups than they did managing the department.

The GAO report does not discuss the power vacuum at the department during the time former secretary Ray Donovan was defending himself against charges -- on which he was eventually indicted -- that he had engaged in illegal business practices when he was an executive of a New Jersey construction firm.

The agency's 194-page report, issued last month, states that about one of every five of the department's employes was fired, demoted or reassigned during Reagan's first term because of a series of budget cuts and reorganizations.

It says the layoffs were often poorly handled and resulted in many qualified employes being fired and replaced by less competent (but more senior) co-workers. Others were transferred to jobs for which they had little background or training, the report contends.

GAO also reports that it found considerable duplication of effort, in everything from hiring to purchasing, among the agencies working under the umbrella of the department.

Labor Department agencies have a wide variety of missions, from coal mine inspections to collection of data used by the government, industry and the courts. Labor Department data are used, for instance, to determine the unemployment rate, increases due persons drawing civil service and Social Security benefits and even some alimony and child payment agreements.

GAO said that Secretary William Brock, who took office earlier this year, generally agreed with its findings that the agency needs to improve its long-range planning, require better program control by managers and reemphasize training programs that were cut back because of budget reductions and layoffs.