San Francisco Case Shows Vulnerability Of Data Networks

By Ashley Surdin
Washington Post Staff Writer
Monday, August 11, 2008

LOS ANGELES -- San Francisco is being forced to overhaul security measures on the computer network that controls data for its police, courts, jails, payroll and health services, as well as other crucial information, after the technology administrator entrusted with the system blocked access for everyone but himself last month and for days refused to reveal the password, even from jail.

Terry Childs, 43, was arrested July 13 at his suburban home, where police found $10,000 in cash, diagrams of the city-county computer network, a co-worker's access card, a loaded 9mm magazine and several loose .45-caliber rounds. Under the user name Maggot617, he hijacked the system and refused to turn over passwords for the network, which superiors belatedly discovered only he controlled. The standoff ended July 21 when Childs relinquished the passwords to Mayor Gavin Newsom in his jail cell.

"I don't want to make it sound hopeless," but "when I go around and give talks, it seems like people don't really understand their risk of being the victim of insider sabotage," said Dawn Cappelli, a specialist in insider threats with CERT, the Carnegie Mellon Software Engineering Institute's Computer Emergency Response Team, which studies security vulnerabilities.

"If you have IT, then it can happen to you."

Childs faces four felony counts of computer network tampering and one penal-code violation for causing losses in excess of $200,000. He has pleaded not guilty but remains in custody in lieu of $5 million bail.

The ordeal has spurred the city's IT department to bolster network oversight and to consider hiring outside auditors to monitor a security upgrade. City officials also will review all access to its FiberWAN network, the hub through which payroll, e-mail and criminal files flow.

It has also persuaded other cities to scrutinize their own systems.

"When these things happen, it forces us to focus on it," said Janis Benton, deputy director of Houston's IT department. "I'm sure we're all looking at this in disbelief. And everybody is going to go back and visit their layers of security."

Such insider threats are a familiar story in the business world, but not so -- at least not publicly -- among local governments. But the scale of San Francisco's cyber-standoff has gained the notice of big-city IT administrators and computer experts, who scratch their heads at how it could have happened.

"It's rare to have a shutdown of this magnitude in a big city," said Edward W. Felten, director of the Center for Information Technology Policy at Princeton University. "This is the type of failure that would only be caused by a major problem -- a major disaster or something like it."

Though all IT departments are vulnerable to attacks from within, most focus on outside threats, Cappelli said. But they should, she said, because in some case, the consequences can be life-threatening.

One city, which she declined to name, faced such a problem when a contract IT administrator altered its 911 system, hoping that fixing it would bring him notice. Consequently, the system that filtered the emergency calls no longer provided dispatchers the caller's address. Cappelli said: "He wanted to look like a hero, but unfortunately he was arrested."

Childs compromised more than 1,100 devices and created unauthorized network doorways, allowing him unfettered and undetectable access. He collected pages of user names and passwords, including his supervisor's, to use their network log-ons. And he downloaded thousands of gigabytes of city data -- possibly privileged information, such as police reports and e-mails -- to a personal encrypted storage device. Experts still aren't sure what data the device contains.

Childs was once among the few administrators to run the network, but his control expanded as colleagues were placed on other projects. The extent of his reach wasn't known until a June computer audit, when administrators found malicious data-eating programs poised on the network.

Afterward, administrators handed Childs suspension papers. He handed them bogus passwords.

The network continued to run, but administrators could not secure or maintain it, posing a potentially time-consuming, million-dollar repair problem should there have been a power failure. So the city spent hundreds of thousands of dollars to restore access.

"It was like we had control of the house, but we were unsure of which rooms he had access to," said Ron Vinson, chief administrative officer for San Francisco city and county's Department of Technology. "We didn't know to what extent he had access or if there were potential vulnerabilities in the system."

Geoffrey C. Bowker, executive director of Center for Science, Technology and Society at Santa Clara University in the tech-savvy Silicon Valley, said he finds San Francisco's management as bizarre as Childs's behavior.

"It's sort of pretty shocking that they didn't have any decent oversight of their own network," Bowker said. "Forget whether this guy was planning any bad actions -- what happens if he has a heart attack and they need to access the system? That's just crazily bad management, but it's not uncommon management."

Not until Newsom made a nighttime visit to Childs's jail cell did he relinquish the coveted password. When he did, Newsom said, "This better be right," according to the San Francisco Chronicle.

Cappelli said that out of 250 insider threat cases CERT gathered between 1996 to 2007 from government and private industry, about 30 percent were IT sabotage cases. Most were premeditated attacks identifiable by red-flag behavior. In addition, 30 percent of the attackers had been arrested previously.

Childs, as it turns out, carried a list of convictions, including aggravated burglary, aggravated robbery and theft, according to court documents. He also served four years in the Kansas state prison. Childs kept this from his employment application, court documents note.

Vinson said San Francisco will probably expand its employee background checks to cross state lines.

© 2008 The Washington Post Company