There is a rather sensational story on the Drudge Report at this moment about an apparent disgruntled network engineer who granted himself god rights on a network, then locked out everyone else’s administrative rights. He then went to jail rather than divulge his password. It’s the equivalent of locking the door and swallowing the key. City Officials claim that the damage caused by this could be in the millions of dollars.

From the SFGate here:

A disgruntled city computer engineer has virtually commandeered San Francisco’s new multimillion-dollar computer network, altering it to deny access to top administrators even as he sits in jail on $5 million bail, authorities said Monday.

Terry Childs, a 43-year-old computer network administrator who lives in Pittsburg, has been charged with four counts of computer tampering and is scheduled to be arraigned today.

Prosecutors say Childs, who works in the Department of Technology at a base salary of just over $126,000, tampered with the city’s new FiberWAN (Wide Area Network), where records such as officials’ e-mails, city payroll files, confidential law enforcement documents and jail inmates’ bookings are stored.

Childs created a password that granted him exclusive access to the system, authorities said. He initially gave pass codes to police, but they didn’t work. When pressed, Childs refused to divulge the real code even when threatened with arrest, they said.

Details on the technical aspects of this story seem murky, and even a bit suspicious to me. From what I can gather, the city of San Francisco recently consolidated its infrastructure under one common network or portal, and they called it FiberWan.

Supposedly Childs was a disgruntled employee who somehow held onto his job despite attempts to discipline him or fire him. Maybe they didn’t want to fire him because he knew so much about the network and were already fearful of losing his knowledge and expertise?

Unnamed “officials” are quoted fearmongering, claiming that the sysadmin sitting in jail had somehow inserted logic bombs to destroy city documents.

Officials also said they feared that although Childs is in jail, he may have enabled a third party to access the system by telephone or other electronic device and order the destruction of hundreds of thousands of sensitive documents.

And more evidence of his mystic digital foo alleges that Childs was digitally monitoring what HR was doing with his personnel case:

As part of his alleged sabotage, Childs engineered a tracing system to monitor what other administrators were saying and doing related to his personnel case…

I couldn’t find out much about Terry Childs, but he is listed as the network POC for all of the city of San Francisco, according to this ARIN registry page here. This means that he had root access to the city’s routers and backbone.

Which leads me to think that Childs didn’t lock down the network at the workstation/server level. He merely changed the passwords on the routers and may have implemented password bruteforcing lockouts so attempts to bypass his password would result in freezeouts.  If this is the case, it hardly makes Childs the criminal mastermind the media is painting him to be.

But this story should be instructive about whom you trust with god-level access.  It should never be a single person.  And if your infrastructure is so critical that a single point of failure can cripple you, you need to identify and eliminate those choke points.