Two-month race to fix HealthCare.gov site hits major deadline Saturday
"War room" mentality takes over as contractors work with administration
"Ruthless prioritization" to get site working quickly may have a consequence
Saturday marks the Obama administration’s self-imposed deadline to fix the troubled HealthCare.gov website, for most people at least.
It caps a frenzied, two-month race by computer programmers and contractors sprinkled throughout the D.C. suburbs on both sides of the beltway in Maryland and Virginia.
The overall scene in response to the botched rollout of the online portal conveys a “war-room” urgency with “ruthless prioritization,” many moving parts, grinding work, briefings, phone-bridges, and of course, plenty of take-out food.
Operations are overseen from a command center in Columbia, Maryland. That’s where contractor QSSI has spearheaded the effort with help from some Web heavyweights.
John Engates, the Chief Technology Officer at Rackspace.com, was an early critic of the website given a behind-the-scenes tour of the effort to fix it.
“I think it’s going to work,” Engates said on CNN’s “The Lead.”
But Luke Chung, president of Virginia-based software developer FMS Inc., called the administration’s prediction the site would work at 80% capacity on or around November 30 an impractical threshold in the software world.
The site, which was taken down for overnight maintenance ahead of Saturday’s deadline, is the lynchpin of Obamacare and is meant to be used by 36 states to enroll millions of Americans into health insurance now required by federal law.
Fourteen states have their own exchange sites and those, by and large, have fared better than HealthCare.gov, which was marred by crashes and other problems upon its launch October 1, embarrassing President Barack Obama, who championed the law known as Obamacare.
What happened in October?
The first thing to figure out was what went wrong – in brief.
“Not all websites are created equal,” Steven Van Roekel, the nation’s chief information officer and a former personal assistant to Bill Gates who helped roll out Windows XP, told reporters on Tuesday at a White House briefing.
Unlike a site like CNN.com, which he said is only to be viewed, or even an e-commerce site, which takes orders, HealthCare.gov is meant to be seamlessly multi-faceted.
A single user’s experience, unbeknownst to them, could require real-time information from the social security administration, DHS, the IRS and Veterans Affairs.
Fixing it on Obama’s orders was ultimately left to Jeffrey Zients, the management consultant and entrepreneur and former and future White House official.
He assembled a team, which includes Van Roekel and other experts, to oversee the overhaul being done by QSSI, other contractors and programmers.
High intensity approach
Zients conducted Tuesday’s White House briefing for reporters at a clipped pace and with the no nonsense tone of a tour guide in a hurry.
The session included a teleconference with the team from QSSI in Maryland – they couldn’t spare the time to make the trip into Washington – and programmers who drove into town from Tyson’s Corner in Virginia, where most of the software work is being done.
Seeking to confound the public view of the site as an epic failure of Presidential management, Zients and the contractors described a hive of trouble-shooters huddled at twice-daily conference calls and on a round-the-clock open phone line to respond as quickly as possible to problems.
“The point of these calls is we don’t have an hour or a minute to waste and therefore, we need to make sure that there is ruthless prioritization at all times as to what matters most, that there’s real-time reaction if something unexpected happened, someone hit a roadblock on a fix – so what are we going to do about that bug and how are we going to address that bug, and you get the best minds on the problems right away,” said Zients.
At a satellite office of QSSI in Virginia, insiders described to CNN’s Brianna Keilar a scene of programmers in marathon code-writing sessions and surrounded by stale Chinese food and half-eaten pizzas.
One result of Zients’ prioritization? A focus over the past two months on the experience for users who want to sign up for insurance. That approach could have a consequence.
Insurers have reported receiving duplicative and at times faulty information on the back end of the site, which raises questions about whether some people who think they’re signing up for insurance will actually have it on January 1.
Indeed, much of the system supporting HealthCare.gov - the back end for making payments and interacting with insurance companies – still needs to be built.
But so far, it has been the difficulty for Americans trying to sign up for coverage in the first place that has partly damaged public perception of the law.
So the programmers and the contractors pay constant attention to real-time site metrics provided by services like Chartbeat, which is used to measure exactly how many people are on the site at any given time.
A quickly flashed screen grab during the technical briefing at the White House showed there were 15,687 users on HealthCare.gov at one point. The aim is to have the site capable of hosting 50,000 at any given time and up to 800,000 per day.
That was the goal for the site back on October 1 when repeated crashes seemed to keep most everyone trying to access the site from viewing insurance options.
After hardware upgrades installed November 23 and 24 to improve capacity and after software programmers spent the past two months addressing other hiccups, supporters of the law hope the website will function as intended.
Awash in metrics
They are awash in detailed metrics at the Maryland command center. In addition to Chartbeat, another web-based program measures response times; a spike on the graph can indicate serious problems. A third measures how far each and every user makes it through the enrollment process.
Don’t look for any of that data after the deadline this weekend, however. Specifics are still closely guarded details.
“We will continue to update you on metrics and user experience and our success rates as we go forward,” said Zients.