San Francisco's city network held hostage

The mainstream media has reported the arrest of the City of San Francisco's network administrator, being held on $5 million bond, as though he had secretly taken control of the city's network and servers and held them hostage, and implies that he has access to data stored on servers on the network. The reality, however, appears to be somewhat different.

Paul Venezia at InfoWorld has dug a little deeper, and found that Terry Childs, a Cisco Certified Internetwork Expert (CCIE, Cisco's top certification), was responsible for managing San Francisco's "FiberWAN" MPLS network, which he, though not the top network architect, built and managed himself. He has always been the only one with access, which he protected vigorously for fear that no one else around him was competent to do so. His paranoia seems to me excessive and misplaced--the risk of no one else having access is itself a single point of failure, and the fact that he originally refused to write remote configuration to flash, meaning that in the event of power failure the devices would not come back up and function properly without intervention, shows him to be a bit off.

Childs never "tampered" with any system or network device to take it hostage, he simply maintained control of what he built and refused to give others access to it. He never has had control of any servers or databases apart from the ones directly involved in managing the network, such as the authentication servers for the network. So the talk of data being stored on the network including "officials' e-mails, city payroll files, confidential law enforcement documents and jail bookings" appears to be irrelevant. Nothing has been done to prevent anyone from accessing any of those things or to gain unauthorized access to them; the network is still up and functioning normally, and Childs didn't have any special access to or manage or control the host-level access to the servers with that data. Now, he was probably able to intercept data transmitted on the network (necessary for troubleshooting), but if sensitive data was only accessed via encrypted sessions, even that risk wouldn't exist.

Childs' problem appears to be that he was overprotective, untrusting of the competence of his peers and management (perhaps with some justification), and placed technological purity and security over business requirements. Not unusual features for people with a very high level of technical skill.

Check out Venezia's article--it looks to me like he's got the goods on this story.

UPDATE (July 23, 2008): Childs gave up the passwords to San Francisco Mayor Gavin Newsom, after a secret visit arranged by his attorney, Erin Crane, with the mayor. Childs' attorney's statements are consistent with Venezia's article:

In her motion to reduce bail, Crane said Childs had been the victim of a "bad faith" effort to force him out of his post by incompetent city officials whose meddling was jeopardizing the network Childs had built. At one point, she said, Childs discovered that the network was at risk of being infected with a computer virus introduced by a colleague.

"Mr. Childs had good reason to be protective of the password," Crane said. "His co-workers and supervisors had in the past maliciously damaged the system themselves, hindered his ability to maintain it ... and shown complete indifference to maintaining it themselves.

"He was the only person in that department capable of running that system," Crane said. "There have been no established policies in place to even dictate who would be the appropriate person to hand over the password to."

The defense attorney added that "to the extent that Mr. Childs refused to turn over the password ... this was not a danger to the public."

Childs intends to fight the computer tampering charges:

Referring to the felony computer-tampering counts, Crane said, "Mr. Childs intends to not only disprove those charges, but also expose the utter mismanagement, negligence and corruption at (the Technology Department) which, if left unchecked, will in fact place the city of San Francisco in danger."

UPDATE (September 11, 2008): Venezia has a new story about the latest round of motions in the Childs case, where the prosecution has filed some apparently technically inept documents. I've also come across an affidavit supporting Childs' arrest from SFPD Inspector James Ramsey (PDF), which presents a very strong case that Childs was up to no good--he had set up his own racks of equipment including modems in a training room, was running his own mail servers and intrusion detection systems, and connecting his own personal equipment to the network. He had cut holes in a locked cabinet next to his cubicle to run cables into them, where he had placed a dialup modem and a computer to allow himself unauthorized access to the city network. The guy seems like a bit of a nut who was engaged in some highly inappropriate behavior meriting termination and criminal prosecution.

UPDATE (August 22, 2009): The judge in the Childs case, Superior Court Judge Kevin McCarthy, has dismissed three charges of tampering, leaving one count related to his initial refusal to give up the passwords, which has a maximum sentence of five years. Childs has served over a year in jail, due to his inability to raise $5 million in bail. He will appear in court on Monday regarding the final charge. Childs gave up the passwords to San Francisco mayor Gavin Newsom after spending eight days in jail.