3 Comments

Summary:

Data center sites and colocation centers in and around New York City are struggling to stay online with varying degrees of success. And there are reports of intermittent issues with undersea cables crossing the Atlantic Ocean.

Updated: Hurricane Sandy continued to take a toll on internet infrastructure in New York City and beyond on Tuesday.

  • There were sporadic issues with undersea cable Atlantic Crossing-2 (or AC-2), sources said. These cables are the main data lifelines between continents.
  •  Telx reported that most of its New York and New Jersey data centers were on generator power as of 9:30 a.m. EDT.
  • As feared, 75 Broad Street, which houses several data centers in the low-lying Zone A of Manhattan, was severely impacted, affecting Internap, Peer 1 and other providers.
  • INIT7, a Swiss provider of IPv6 infrastructure, was affected by a storm-related power outage at Equinix’ 8th Avenue facility in Manhattan.  The company also reported connectivity issues to Miami and Los Angeles that have since been resolved.
  • Equinix reported widespread issues with its data centers in the areas around NYC, but said they all have 48 hours of fuel
  •  A Navisite data center in Manhattan’s Zone A is also running on generators. It has refueled and has enough to last 72 hours and will refuel as needed.

Lobby at Verizon office at 140 West Street, New York.

The specter of trouble with the undersea cables could be a huge deal, although experts said that there is so much redundancy that much of the risk is mitigated.  There were some reports attributed to cable operators who said they had experienced power issues but back-up generators prevented service disruption.

Update: (5:30 p.m. EDT) Level 3 Communications, just emailed a statement about the status of the AC-2 cable:

“We made extensive preparations in advance of the storm and have experienced no major service disruptions. All of our subsea cable systems are operating normally. We have experienced a minor fiber cut to one of our diversely routed, secondary backhaul lines which provides connectivity to our AC2 Cable Landing Station, and our technicians are currently working to repair it as quickly as possible. Few customers were affected because most traffic was rerouted through redundant lines. And again, the AC2 subsea cable itself is operating normally.”

Carriers and ISPs use the cables to pump data across the Atlantic. Two of the major gateways are in Brookhaven, N.Y., (on Long Island) and in northern New Jersey, where the cables come ashore. “Given the duration and strength of this storm, you have to worry about the cables getting disrupted,” said an executive with a data center company who did not want to be named. He said problems with the cables is “ the kind of thing companies will keep close to the vest. If  either AC-1 or AC-2 were to go out there would be major, major issues.”

As already reported, data center facilities in lower Manhattan suffered a string of outages after flooding and Con Ed cut electrical power. Datagram, the web hosting company that serves the Huffington Post, Gawker, Gizmodo and BuzzFeed, went down Monday evening after flooding caused those sites to go dark. Data centers at Google-owned Carrier Hotel on 8th Avenue, including Equinix, XO Communications and others were also reportedly affected by power outages, though it appears some are back up and running. And Atlantic Metro Communications also reported disruptions due to flooding at a New York data center.

Ryan Kim contributed to this post.

Cable map courtesy of Telegeography; flood photo courtesy of Verizon.

You’re subscribed! If you like, you can update your settings

  1. Gordon Chumway Tuesday, October 30, 2012

    Great information

  2. Brandon Greenberg Tuesday, October 30, 2012

    Reblogged this on Brandon Greenberg and commented:
    This article highlights the importance of making a conscious decision about where you house your data. While I firmly believe the cloud is the best place to be, you should probably have your data sitting outside your geographic region.

  3. geographically disparate replicated infrastructure always make the most sense for true uptime requirements.

Comments have been disabled for this post