Obama administration officials have warned that ambitious experiments run by the Affordable Care Act’s $10 billion innovation lab wouldn’t always be successful. Now there is evidence their caution was well-placed.

Only a small number of community groups receiving federal reimbursement to reduce expensive hospital readmissions produced significant results compared with others that were not part of the $300 million program, according to partial, early results.

The closely watched project is one of many tests to control costs and improve care being done by the Center for Medicare and Medicaid Innovation. The center was created in the health-care law.

Dozens of community agencies on aging, from Ventura County, Calif., to southern Maine, were offered money to try to ensure that seniors leaving the hospital received care that reduced their chances of being readmitted within a month.

But an early evaluation found that only four groups out of 48 studied in the Community-based Care Transition Program significantly cut readmissions compared with those of a control group.

Also, 29 groups have withdrawn from the program or been terminated by the Department of Health and Human Services for failing to achieve targets, agency officials said. The CCTP project, which has grown since the early evaluation, has 72 participating sites that administration officials hope will yield readmission reductions and lessons in post-hospital care.

The evaluation, conducted by consulting firm Econometrica under an HHS contract, is one of the first independent analyses of an innovation-lab test to be made public. The 111-page report is dated May 30, 2014, but was posted Jan. 2 on the HHS Web site.

Still, experts said it’s too soon to evaluate CCTP’s effectiveness.

“It’s really too early to tell,” said Ellen Lukens, who leads the practice on hospital and post-hospital care at Avalere Health, a D.C.-based consulting firm. “Can you really evaluate this when it’s been such a short period of time?”

A five-year experiment, the program signed its first round of deals with community agencies in 2011 and its fifth and last round in March 2013. Econometrica’s report covered partial 2012 results from groups participating in the early rounds, including some for which only a few months of data were available. Congress required the lab to monitor all tests closely.

One positive note was that “a lot of the sites were able to implement the program very quickly,” Lukens said, adding that later data will give a better idea of CCTP’s effectiveness.

The readmissions result — less than one site in 10 significantly reduced them — “seems kind of wimpy,” said Eric Coleman, a University of Colorado professor whose previous work on care for discharged patients influenced the CCTP program. He said he remains optimistic about the tests, noting that the results are early. He praised HHS for cutting out nonperforming groups.

“This is really the first glance of the first two waves of the program,” HHS spokesman Raymond Thorn said. “It’s too early to determine whether this model is failing or not. We will have successes.”

CCTP is one of dozens of experiments being run by Health and Human Services’ innovation lab, which has a 10-year, $10 billion budget. Preventable readmissions are estimated to cost the Medicare program $17 billion a year.

Paying community agencies to work with hospitals was considered one potential way of reducing readmissions. Rather than getting grants, agencies are paid according to the discharge cases they handle.

The program faces several challenges, experts said. For example, HHS favored funding groups working with hospitals that have high readmission rates, making success more difficult.

Additionally, numerous groups and hospitals are working to cut readmissions through other means. That increases competition for agencies trying to make their mark and raises the difficulty of measuring their results separately from other programs.

Readmissions have been dropping nationally since Medicare in 2012 began penalizing hospitals that logged too many repeat patients. Some CCTP groups reduced readmissions — but so did comparison hospitals. That means the system improved overall in those areas and money was saved, but statistically the agencies for the aging did not show up as the critical factor.

Coleman faulted HHS for requiring agencies to file detailed reports on care models and administration rather than letting them focus on the main job.

“If it doesn’t reduce readmissions, it’s game over, so why do you want all these process measures?” he said. “If we want these sites to succeed, we need to get out of their way.”

More than 100 agencies originally agreed to participate. But 29, including New York Methodist Hospital and Pennsylvania’s Delaware County Office of Services for the Aging, have withdrawn or did not get contract renewals because they missed readmission-reduction or enrollment targets, HHS said.

A list of agencies that have left the program is available with this story online.

The health-law innovation program also includes accountable-care organizations to cut costs and improve care quality; tests giving more resources to primary-care doctors to coordinate care; and innovation awards for models designed to improve Medicare efficiency.

Administration officials like to compare the lab to a venture capital fund, in which many investments are expected to fail but a few succeed spectacularly. Many Republicans think the lab is a waste.

— Kaiser Health News

Kaiser Health News is an editorially independent program of the Kaiser Family Foundation.