Although the Reagan administration expected to save money by allowing Medicare patients to enroll in prepaid health maintenance organizations (HMOs), a study has found that Medicare is paying HMOs too much in patient premiums and that the government may lose money on the arrangement.

The draft report by the General Accounting Office said Medicare HMO enrollees are healthier than the general Medicare population, primarily the elderly, and suggested that HMOs could still make money with a lower premium. The Department of Health and Human Services had no comment on the report.

The report, prepared at the request of Rep. Lawrence J. Smith (D-Fla.) and based on studies of 27 HMOs in which Medicare patients were enrolled in 1984, casts doubt on the administration's broad strategy of encouraging Medicare patients to sign up for prepaid HMOs that provide them the full range of Medicare benefits in exchange for a single, fixed annual premium paid by the government.

Under a 1982 law, the premium is set at 95 percent of the amount the government would expect to pay in regular Medicare benefits for the average person that age if he or she stayed in the regular Medicare program.

By setting the premium payment to HMOs at 95 percent, the government figured to save 5 percent. But the GAO said Medicare patients who signed up for HMOs were the healthier ones who would not have cost the government much if they had stayed in the regular program. The GAO concluded that to achieve a real savings, the premium paid to the HMOs would have to be reduced to 89 percent.

The GAO did not have a full explanation of why Medicare patients who shift to HMOs have less illness, but it found that "HMO enrollees were healthier than the average beneficiary as measured by mortality rates. They would generally need less medical care and cost the HMOs less overall."

The GAO said it found that mortality rates for Medicare beneficiaries enrolled in HMOs were only 74 percent of their projected level adjusted for age, sex and other factors -- so the enrollees, in effect, were healthier than the overall Medicare population.

The GAO found a lower percentage of HMO enrollees were institutionalized and a lower percentage were at low-income levels than in the general Medicare population.

Although the GAO study based its broad reimbursement conclusions on 27 HMOs with Medicare enrollees, the report, first disclosed in the Fort Lauderdale News & Sun-Sentinel, looked in detail at four southern Florida HMOs with a combined enrollment of 155,857 Medicare enrollees as of Dec. 31, 1985. The four HMOs included International Medical Centers Inc., with 134,803 enrollees, which has just been notified by HHS that it must revise many of its procedures.

The report said as of March 31, 566,190 Medicare beneficiaries nationwide had enrolled in HMOs.

Overall, HMOs had 21,051,657 non-Medicare and Medicare enrollees at the end of 1985 -- a 25 percent increase over 1984 -- according to InterStudy, a study group that takes an annual census of HMOs.

Looking at the four Florida groups, the GAO report said that many subcontractor medical groups providing services to the HMOs "did not have to meet federal requirements concerning financial solvency and enrollment." The report also said there was inadequate federally sponsored peer review to monitor performance, and that there was "potential for HMOs to screen applicants to enroll only healthier individuals."

"This report documents what I've said before -- that the overview of the HMO Medicare program has been very poor on the part of the government," Smith said. "That is why I opposed opening up the program around the country. In two to three years there will be millions of new members on Medicare with no real system of monitoring quality of care, the financial status of the HMOs, and if the government is saving money.

"Because the budget for HHS has remained stagnant and personnel is being reduced," Smith said, "there will not be enough people to manage the increased workload involved. As time goes on it will only get worse.

A Group Health Association of America spokesman said, "We are aware that the reimbursement formula is imperfect, but we don't think that the methodology employed by the GAO justifies a 5 percent reduction in the reimbursement formula . . . . More evidence is necessary before we can accept these conclusions."