[qi:066] Yesterday afternoon, I left my apartment, which also doubles as my part time office, in a mad rush, late for a meeting. As the elevator descended, the lights suddenly went out, and for a minute or two (it seemed like an eternity) I was suspended in a dark metal box. And then survival skills took over, as I pressed the emergency button.
And while on the phone, the lights came back on, and the elevator descended, and I rushed to my meeting, silently cursing the building owners for having shoddy infrastructure. It was only later in the day, I learned of massive power outages caused by snafus with at PG&E, our local power company. This resulted in a transformer blowing up, and causing even more disruptions, especially at 365 Main, one of the large co-lo/data center facilities situated in the SOMA area of San Francisco.
This resulted in massive outages at some of Web 2.0’s brand name companies – Six Apart, Facebook, Technorati and Yelp – knocking out their systems and web services out flat. Whatever the reasons behind the failure might be, yesterday was a rude reminder of how fragile our digital lives are.
The seemingly invincible web services (not to mention the notional wealth they signify) vanish within a blink of the eye. It was also a reminder, that all the hoopla around web services is just noise – for in the end the hardware, the plumbing, the pipes and more importantly, the power grid is the real show.
According to North American Electric Reliability Council (NERC) there has been a 50% decline in the capital expenditures by utilities over the last 15 years. The underground cables are crumbling. This report in Fresno Bee takes PG&E to task.
… San Francisco consulting firm KEMA Inc., which wrote in a report two years ago that PG&E distribution equipment “is getting older both in terms of average age and the percentage of very old equipment. … PG&E increasingly will have to become more proactive in addressing issues related to aging equipment.”
This is not just a problem with PG&E. Power grid across the country is aging. It is ironic because data centers/hosting business is one where US companies still dominate. Thanks to an abundance of long haul bandwidth and ample data center capacity, many overseas companies find it more secure to host their digital infrastructure in the US. Bebo might be big in Britain, but its websites are served up from 365 Main.
The reliable data center/colo facilities have been a distinct advantage for US start-ups, especially the Web 2.0 start-ups. And yet we continue to bet our future on this creaky house of cards. That’s like building a Taj Mahal on quick sand. And that is one sinking feeling – the same one I got in the 100 odd seconds I found myself stuck in the elevator to nowhere this afternoon.