The World Trade Center attack threw a spotlight on companies whose sole purpose is disaster recovery. Their services include the duplication of all incapacitated datacom and telecom infrastructures and end devices. While the scale of loss on September 11 eclipsed anything the two most prominent companies, Comdisco and Sungard, had ever dealt with before, it did show how preplanning had paid off for some of their clients. It also put into action agreements made well in advance between enterprises and long-distance carriers. These included rerouting plans, so that as soon as disaster declarations were made, the new emergency locations were already on file as the destination of rerouted 800 and other phone numbers.
Regular systems integrators and interconnects, under standard maintenance contracts, also took up the task of recovering dial tone for New York City's emergency services and for dislocated - or, as often, disconnected - financial and healthcare clients. This required round-the-clock, emergency enlistment of designers, technicians, and installers from offices as far as Maine; the rerouting of trunking switches to New Jersey locations, and even the cooperation of Canadian customs, which expedited the importation of Nortel switches.
Nortel Networks (Brampton, Ontario, Canada - 905-863-0000, www.nortel.com), maintains a year-round, 24/7 Emergency Recovery Center in Research Triangle Park, NC, that normally employs over 100 people. When disaster struck, Nortel's ER immediately contacted all customers in the World Trade Center vicinity to determine stability and to analyze any service or equipment requirements. These calls were then escalated to Nortel's service and support organizations (Nextira among them -- see below for more).
Tornadoes have taken out central offices before, and Nortel's ER has also responded to floods and fires, but obviously, no previous damage has ever been as extensive as that wrought in the September 11 assault. But having a plan of action, numbers to call, and a recovery center certainly did help dislocated enterprises meet immediate needs.
By Monday, September 17, Nortel Networks reports that 432 orders had been loaded and serviced with a 48-hour turnaround from order to on-site; 34 customers had been serviced, including carriers Focal, Genuity, and Sprint, as well as Goldman Sachs, Bloomberg, and the Pentagon.
In Verizon's heavily damaged, 32-story 140 West Street central office, two Nortel DMS 100 switches, servicing 60,000 of the office's 300,000 lines, went down when power went out and basement power supplies were flooded. Nortel's team was allowed to enter the building on Friday, escorted by security offers and wearing respirators. They worked 18-hour shifts and often had to evacuate over the weekend as fires flared up and areas were cleared from reported bomb threats.
The Nortel switches proved to be fortunate, as they were located near the wall furthest from the damage caused by the toppled neighboring Building Seven. According to Denny Miller, director of emergency recovery, "They had some damage, but nothing we couldn't work around. We had those backed up with backup generators and temporary air conditioning units trucked in by September 18." Verizon sealed off the part of the floor where the DMS switches were housed.
Nortel ER teams also worked with optical MAN specialists Metromedia Fiber Network (www.mmfn.com) to provide Optera Metro 5200s multi-service platforms on Optical MANs and fully restore service by Wednesday. With AT&T, it worked to restore Merrill Lynch's optical ring service between New York City and New Jersey.
Doubling up in branch offices
Employees of Nextira (Houston, TX - 800-324-2222, www.nextira.com), while not in the disaster recovery business per se, found themselves in the thick of the New York recovery effort. A major systems integrator for Nortel, Nextira is the convergence solutions company owned by Platinum Equity Holding and founded from the merger of the spun-off Williams Communications Solutions, Timeplex, and Milgo. Nextira employees worked in central offices for Verizon and colocated local and long-distance carriers, and also on behalf of enterprise clients.
Interviewed ten days after the attack, Frank Pilon, Nextira executive director of operations for eastern territories, says, "We're seeing a lot of expansion on existing CPE switches where companies are moving dislocated personnel to double up in other, nearby offices." In other places, calls and dial tone were being routed off remote switches connected via T-carrier fiber.
On the day of the World Trade Center attack, Nextira established an emergency command center in Connecticut and a secondary staging area in New Jersey for customer support, rerouting voice and data networks. One day following the attack, the company had successfully restored communications networks for the City of New York Command and Control Center, Nasdaq, and Merrill Lynch.
New York City Command and Control Center
The City of New York Command and Control Center had been housed in Building Seven of the World Trade Center complex, which was the fourth building to collapse after the Twin Towers. "Our New York City office got a call from Nortel at 6:30 or 7:00 Thursday night," comments Frank Pilon. "They said they had to have 300 phone lines in the relocated New York center, by the next morning. So we pulled a team together, scrambled, got design specialists to work around the clock. We brought two Option 11 Meridians from a warehouse in Maspeth, Queens. We networked them together and by 6:30 in the morning on Friday we had 280 lines wired and ready to go. We just had to wait for Verizon to do their work, which they had up by later that morning. Usually that would have been a three-week job." There's now a third Option 11 and 300 lines; all three cabinets are to be replaced with one Option 81.
It ordinarily takes 15 to 16 hours to build an Option 81 on order, says Pilon. Then it has to be shipped from Canada and cross customs, which typically takes 50 to 60 hours. In the disaster relief effort, the switches took 36 hours after order to arrive.
Nextira also restored critical communications functions for NASDAQ over the weekend. The exchange has squeezed workers dislocated from the WTC into preexisting space at other Manhattan offices. At 125 Broad Street and Four Times Square, Nextira installed Nortel PBXs. At 33 Whitehall Street, Nextira installed an additional Nortel Option 11 Meridian 1 and several hundred phones over a weekend, according to Pilon.
NASDAQ locations were also handling calls (even at presstime) for the disabled American Stock Exchange.
For Merrill Lynch, Nextira served in its role as maintenance subcontractor for British Telecom. They replaced a British Telecom switch and some of the turrets - special trading-desk phonesets - lost in the disaster. The first 200 of the 800 turrets and the switch went in overnight in Merrill Lynch's new office space in Jersey City, NJ. Five hundred were ringing by the following Monday.
