Administration officials are preparing to announce Sunday that they have met their Saturday deadline for improving HealthCare.gov, according to government officials, in part by expanding the site’s capacity so that it can handle 50,000 users at once. But they have yet to meet all their internal goals for repairing the federal health-care site, and it will not become clear how many consumers it can accommodate until more people try to use it.
Government and outside technical employees worked through the night on the latest upgrade, intended to increase the Web site’s capacity for consumers seeking to go through the early stages of registering for an account and then logging in. The upgrade was successfully completed about 4 a.m. Saturday, according to a government official familiar with the project, who spoke on the condition of anonymity to describe details that the administration was not publicly disclosing.
But it was not immediately apparent, the official said, whether the improvement meant that the site can now meet one of the Obama administration’s internal goals — for 80,000 people per hour to be able to register and 320,000 per hour to be able to log in — and added that the overnight tinkering was, at least for now, causing a slight increase in error messages on the site.
An official at the Centers for Medicare and Medicaid Services, the agency overseeing the federal health insurance exchange, said the site’s true capacity is somewhat murky because workers need to see how it performs under “weekday traffic volumes” when demand is at its peak.
Nov. 30 was not originally intended to be a key date for the online enrollment system, but it took on outsize political and public importance when administration officials announced five weeks ago that the “vast majority of users” would be able to sign up for insurance through the site by that day. A combination of federal employees, outside contractors and a handful of technical and management experts have worked at breakneck speed for five weeks to improve the Web site’s performance as the White House has come under withering criticism from its political opponents and some consumers.
In an interview with Barbara Walters that aired on ABC on Friday night, President Obama said he was confident that, in time, Americans would come to embrace his controversial health-care law.
“I continue to believe, and [I am] absolutely convinced, that at the end of the day, people are going to look back at the work we’ve done to make sure that in this country you don’t go bankrupt when you get sick, that families have that security,” Obama said. “That is going be a legacy I am extraordinarily proud of.”
Administration officials have said for several weeks they define success as having “the vast majority of users” be able to navigate the site and sign up for insurance. While they initially did not define what that meant, White House press secretary Jay Carney said earlier this month that the administration’s aim was to have 80 percent of users enroll through the site. Those working on the project have set speed and error rates as a way of measuring that goal.
A significant upgrade in the Web site’s capacity was carried out early this week, according to the government officials, which allowed HealthCare.gov to accommodate more people at once without causing the site to malfunction. For instance, many more consumers can simultaneously be on a section of the site designed to let people compare available health plans.
Even so, how much load the site can withstand depends on how much consumers are asking it to do. For example, simply looking around on the “learn” part of the site — where people can get basic information and a rough sense of prices — puts less strain on the system than the areas where people actually apply for coverage.
The team has not yet attained its target of cutting the average load time across the site to no more than half a second, officials said, though it is running faster than before.
The blitz to rehabilitate the site has entailed an unusual public and private partnership over the past five weeks. The White House rejected the idea of turning over the work of fixing the site mainly to Silicon Valley allies, some of whom served as the president’s campaign operatives, deciding instead to selectively tap a handful of them to help steer the project.
Michael “Mikey” Dickerson, a Google employee who is on leave and volunteered on both of Obama’s campaigns, has led the twice-daily calls in which the entire team has discussed ongoing problems and how to solve them. Civis Analytics chief technology officer Gabriel Burt, who served as lead analytics engineer and product manager for Obama’s 2012 reelection campaign, has also been deeply involved.
Jeffrey Zients, the former acting director of the Office of Management and Budget who was designated by the White House in late October to oversee the project, told reporters during a briefing this week that the administration “brought in a fresh set of eyes” the week of Oct. 21 to examine the system. He said that survey concluded within a few days that the Web site could be fixed and that it did not need to be thrown out and a new one installed — but that the fix would require “additional talent and expertise” and a different way of approaching the site’s defects.
The work on HealthCare.gov took place primarily in two places: in a command center dubbed the “Exchange Operation Center,” or XOC, in Columbia, Md., and in the Tysons Corner area of Fairfax County. QSSI President Bikram Bakshi — whose company developed the system’s data hub and became the project’s general contractor on Oct. 25 — said in the briefing that “other business analysts, system analysts, developers and testers [were] scattered across the D.C. metro area.”
Since starting work, the team has “knocked through well over 300 bugs,” Zients said, and each evening the group has either put in hardware upgrades or fixed a glitch during the four- to five-hour maintenance window it has to work on the site.
The 25 people working in the command center, who represent all the major contractors that helped build the site, can evaluate HealthCare.gov’s real-time performance through 15 screens capturing its overall response time, error rate, volume of traffic and visitor makeup and which functions users are performing at a given time. When they see a disparity in the readings, Bakshi said, “then the team springs into action” to try to fix it.
John Engates, chief technology officer for the cloud computing company Rackspace, toured the project’s command center Monday with a handful of other private tech experts and said he was encouraged that workers there have streamlined the decision-making. “There isn’t a lot of ‘Let’s run it up the flagpole.’ That’s what’s helping things run more smoothly,” he said.
Before touring the command center, Engates, along with representatives from Salesforce.com, IBM and other firms, met in the Situation Room with senior White House officials, including Zients, Chief of Staff Denis McDonough, chief technology officer Todd Park and chief information officer Steven VanRoekel.
Salesforce.com chairman and chief executive Marc Benioff, who hosted a fundraiser with Obama at his home Monday in San Francisco, said in a phone interview that he has dispatched two employees to the White House to provide what he described as “coaching and mentoring.”
“A friend in need is a friend indeed. Silicon Valley is a friend of President Obama’s, and we’re going to help the government get through this,” Benioff said, noting that his employees were taking no compensation for their efforts.
But there are also limits to this support. Benioff said he had “offered to rebuild the entire HealthCare.gov site for free and run it for five years at no charge” in direct talks with the White House, but the administration declined the offer.
A White House official said the administration could not have accepted the offer, because it ran afoul of federal contracting rules.