Millions of poor, minority and handicapped youths are being cheated out of a quality education because of "inhumane" budget cuts by the Reagan administration and "unfair" public school policies, a national panel of children's advocates charged yesterday.

Headed by a former U.S. commissioner of education and the president of the Children's Defense Fund, the panel issued a report that said the nation's push to raise standards neglects the special needs of children "at risk" and could lead to a "permanent underclass in America."

Harold Howe, education commissioner under President Johnson, described the Reagan administration as "antichildhood" and said there is a trend in many states to put a disproportionate amount of funding in schools located in middle-class and wealthy neighborhoods. He said this trend is tantamount to a "conspiracy" to lock poor children into an economic underclass.

The report, titled "Barriers to Excellence: Our Children at Risk," is a response to "A Nation at Risk," a report that was done by the National Commission on Excellence in Education and released by the White House in 1983. That report recommended more rigorous academic instruction but did not urge increases in federal funding for special programs, Howe said.

Most "at risk" children "need extra help to attain the levels of learning of which they are capable," but progams that provide such help are being damaged by Reagan's conservative fiscal policies, Howe said.

Howe and Marian Wright Edelman, president of the Children's Defense Fund, headed a board of inquiry that conducted a two-year study for the National Coalition of Advocates for Students, a Boston-based network of 18 child-advocacy groups in 15 states. A compilation of public hearings and research conducted in 10 cities located in such states as New York, Mississippi and Texas, the report urges the federal government to increase the funding of the Chapter One Program, which provides compensatory services for poor, minority and handicapped children.

It also recommends that the program be expanded to the secondary level to provide "dropout prevention" and that the Head Start program be strengthened.

According to the report, public school systems across the country use "tracking systems" that place whites in advanced classes and minorities in slower ones; discriminate by race, class and sex; train students to take standardized tests while ignoring their other needs; label three times as many blacks as whites as "educable mentally retarded," and suspend a disproportionate number of black students for "no discernible reason."

The report also urges state governments to "recognize that federal funds for compensatory education are 'add ons' that do not relieve states of their basic responsibility to provide these services and allocate state funds accordingly."

Howe characterized the "Nation at Risk" report as rhetoric without substance because it "speaks of all children," but does not address programs needed to bring poor students up to higher standards. "It was a bugle call . . . for greater academic rigor, but does not recognize that some 20 to 25 percent of students. . . need very special help. . . and that that help is going to cost money and is going to take changes in schools and is going to take a long time to bring about.

"We believe that many of our youth who fail in school are more likely to be rescued by the suggestions we have made than by prescriptions for greater rigor unaccompanied by the combination of adequate funds and more personalized caring," Howe said.

Edelman said the report is the first step in an effort to persuade members of Congress to oppose additional spending cuts for public schools. "Many members of Congress supported Reagan's cutbacks with some reservations . . . and they now see that this administration has gone too far and has hurt many children."

She said she was optimistic that "with greater awareness and greater advocacy, the Congress will reject further cuts and indeed give us back what we have lost."

The 162-page report, funded by the Carnegie Corporation, the Ford Foundation and other sources, says that in many states school systems place barriers "in the way of student learning" by "segregating" disadvantaged youths from the mainstream of teaching.

According to the report, most "children at risk" don't master basic skills and drop out of school. A lack of attention to the needs of pregnant teen-agers and inadequate job opportunities present major obstacles to students' abilities to stay in school and develop the motivation to pursue education enthusiastically, the report states.