A major study by sociologist James S. Coleman concludes that Catholic and other private high schools provide a better education than public ones do, and in some respects are less racially segregated. The study is certain to cause controversy in U.S. Educational circles.

Coleman, whose 1966 report on educational opportunity became one of the decade's most debated documents on schools, based his latest findings on data covering 58,728 students in 1,015 high schools.

The report was compiled for the National Center for Education Statistics, a branch of the Department of Education. It is one of the most extensive examinations of non-public schools ever sponsored by the federal government.

Although officials of NCES described the report yesterday as a "draft . . . still subject to change," it has been sharply attacked by supporters of public schools who fear that the findings could strengthen the case for tax credits to families paying tuition for private schools.

Coleman went on record in favor of tuition tax credits as early as 1978, saying they would widen the school options of low-income black parents.

President Reagan has promised to fight for tuition tax credits that would aid families that sent their children to parochial or private schools. But a major storm is brewing over this, with public-school backers saying that Reagan's policy is to slash funds for schools serving poor and minority children and transfer the savings to middle-class families through tax credits.

David Sweet, assistant administrator of NCES, said yesterday that "we would not say we fully support each and every conclusion" reached by Coleman. And an internal NCES evaluation of the Coleman study obtained by The Washington Post "urges the author(s) to take less liberty with the data and provide a more complete picture" of one of the issues dealt with.

"The statistical techniques used here are out of data, and I think nearly every expert in the field would say that the method has to overestimate the quality of private schools," said Robert L. Crain, an educational sociologist and author who now works at the Rand Corp. "If you took a high school student body which came from families of above-average income, those kids would do well in either a public or private school. There is no evidence in this report that makes me think they would do any better in private school."

Coleman, a University of Chicago sociologist, is affiliated with the National Opinion Research Center, which has the contract to gather data for a massive federally financed survey of high school sophomores and seniors.

About one high school out of four in the United States is non-public, but only about one of 10 students in grades 9 through 12 attends these facilities. By far the largest number of the private school students -- 900,000 of 1,359,000 -- attend Catholic parochial schools.

To gather the data, the National Opinion Research Center examined school records, tested thousands of students and interviewed administrators.

According to Coleman, "the evidence is that private schools do produce better cognitive outcomes than public schools. When family background factors that predict achievement are controlled, students in both Catholic and other private schools are shown to achieve at a higher level than students in public schools."

In a warning that critics already have seized on, however, Coleman acknowledges that factors other than family background could affect achievement.

"Coleman has mixed up cause and effect," a critic said. "These kids are in Catholic schools because they have high scores. They don't have high scores because they're in Catholic schools. This is not a minor problem; it's huge."

Also certain to provoke controversy is Coleman's finding that "Catholic schools more nearly approximate the 'common school' ideal of American education than do public schools," in that achievement levels of children with different backgrounds are more nearly alike in Catholic than in public schools.

Coleman also challenges a major contention of critics of tuition tax credits, who assert that the aid would intensify racial segregation.Coleman argues, on the basis of computer models, that there would be a slightly higher percentage of blacks and Hispanics in private schools after the first year of such aid than are there now.

He argues that this would be constructive because "the public sector is by far the most segregated" on a school-for-school basis.

In his analysis, Coleman concluded that individual private schools are less likely to be racially segregated than individual public schools, although public schools have a considerably higher percentage of black enrollment than Catholic schools nationally. Coleman reports that public and private schools now enroll about the same percentages of Hispanics.