The Commerce Department said the cyberattack on its job-development bureau was so vicious that the agency’s entire computer network had been put at risk.
Federal officials were so worried the infection would spread to economically sensitive information that 200 employees had to spend months without e-mail or access to Internet servers and databases. BlackBerrys were abandoned, and there was no Internet communication with regional offices. Officials spent almost $3 million to destroy computers, hire consultants and secure temporary networks before building a new operating network from scratch.
Now comes news that the attack was no attack at all.
The Commerce Department’s inspector general has determined that the supposed attack that crippled the Economic Development Administration for almost a year in 2012 was nothing of the sort. The disruption turned out to be a common malware infection on six computers that could have been erased with anti-virus tools and other security steps.
“EDA’s persistent, mistaken beliefs resulted in an excessive response and ultimately unnecessary expenditure of valuable resources,” Inspector General Todd J. Zinser wrote in a recent report.
“There was no evidence to suggest that EDA’s primary business application had been targeted by a cyber attack or maliciously altered,” he wrote.
Zinser described a series of errors and miscommunications that led the EDA to take drastic steps immediately after a Department of Homeland Security team flagged a possible virus in December 2011.
Inexperienced, unqualified IT employees overreacted to information that turned out to be wrong, investigators found in their June 26 report. They spoke past one another and did not validate how many computers had been targeted. They failed to heed early conclusions that this was not a large-scale attack by a foreign entity.
EDA officials destroyed desktops, laptops, servers and printers worth $175,000. The destruction stopped only because they ran out of money — and Commerce officials denied their request for millions of dollars to demolish more equipment.
EDA officials hired an outside cybersecurity contractor $823,000 to investigate the attack and spent $688,000 to come up with a long-term fix for a problem that didn’t exist.
They wasted $1.1 million on new computers and other temporary equipment they are now replacing with permanent networks.
The expenses came to half the department’s technology budget. In an environment of heightened vulnerability to cyberattacks, “once you’re infected you often overact,” said Alan Paller, research director of the SANS Institute, a cybersecurity training school in Bethesda.
“You feel violated,” Paller said. “All you feel is somebody’s in my house and I’ve got to get them out. And you get overly conservative.”
Rebecca Blank, appointed as acting commerce secretary in June 2012, asked Zinser to investigate the response.
“The EDA did not know what it was facing,” said Blank, who left the Obama administration this spring. “Under those circumstances, given the cyber risks [to the government], one has to be cautious.”
“In retrospect, it was not as serious as they originally thought,” she acknowledged. “But it’s a question of which side do you want to err on?”
In a statement, the EDA said much the same, adding that the department’s mission — giving business-development grants to distressed communities — continued with “excellent customer service.”
Commerce officials said they have addressed the fragmented IT system cited by the inspector general by sharing systems with small departments, including the EDA. They declined to say whether any employees have been disciplined.
The response quickly went off the tracks. An incident response team at Commerce that responds to network problems gave the EDA’s IT staff wrong information, telling them that more than half of its computer networks were infected.
Commerce officials corrected their error the next day, describing just two faulty computers. But the EDA misunderstood the new notification, and the response team at Commerce did nothing to make sure the accurate information had gotten through.
The botched communications persisted for almost a year, until Zinser’s office put the pieces together.
The National Oceanic and Atmospheric Administration, also part of the Commerce Department, was informed of a possible cyberattack about the same time. NOAA cleaned up a small number of infected computers within a few weeks.
But at the EDA, fear of foreign cyberattacks was so high that the department called in help from the Homeland Security and Energy departments, the National Security Agency and a private cybersecurity contractor.
All reached the same conclusion: There was no widespread malware infection.
But the EDA pushed ahead with its costly recovery plan, cutting off employees’ e-mail and Internet access and smashing computers.
Zinser determined that the Commerce IT employee who handled the case did not have enough training to accurately diagnose the problem.
The ostensibly infected e-mail server was, in fact, protected by up-to-date anti-virus and other security protections. And even if there had been an infection, it could not have spread to other parts of the network because outbound e-mail traffic does not pass through other Commerce computers, investigators found.
Complicating matters was Commerce’s history of poor system security. EDA’s chief information officer was relatively new to the job and based his assessment partly on that assumed vulnerability, investigators found.
Fear “led the Office of the Chief Information Officer and EDA not to question the accuracy of the extent of the malware infection, despite a lack of supporting evidence,” Zinser found.
It wasn’t until March that operations were fully restored.
The inspector general recommended that the EDA “not destroy any additional IT inventory that was taken out of service as a result of this cyber incident.”