Here’s yet another article that I’ve excerpted from the Wall Street Journal on how Hurricane Sandy reminds us of the need to plan for situations when our normal course of business is interrupted — and for disaster recovery (DR) sites to truly be geographically diverse (read: far enough away to be beyond the reach of a single weather or geologic event).
Good news: Phoenix and San Diego are far enough apart to qualify as respective DR locations.
I had to chuckle when reading the article, too: the thought that came to mind during the part about Freshpair.com (an undergarment retailer) was, “Gosh, they’ve been caught with their pants down.” Groan.
I also remarked to myself that not only did MailChimp “dodge a bullet”, they also “nearly slipped on a banana peel” that could have really brought them down hard. Double-groan.
Emphasis in red added by me.
Brian Wood, VP Marketing
Natural disasters of the past few years taught companies the importance of good backup systems —but Hurricane Sandy showed many that there is still a lot to learn.
The good news is that many businesses spared themselves a great deal of economic loss by keeping duplicates of their data, email systems and other applications far from their primary data centers. Employees were able to access those systems from their homes or elsewhere via software and mobile devices.
Still, many companies have struggled to cope with the effects of the storm. Some found that their critical systems weren’t in fact backed up, or that their duplicate systems weren’t located far enough apart to provide the desired diversity. Others learned their plans were hampered by the inability of their employees to access those systems via the Internet, or to travel to alternate locations that needed tending.
Freshpair.com, a New York-based 50-person retailer of undergarments, has been able to maintain most of its operating capability. Most of the company’s applications, including email, tools for managing paid online search programs and business analytic software, are running in remote locations across the country.
But its software-based phone system, which includes a feature that lets employees route calls to their home or cellphones, isn’t duplicated at a remote location. Because its headquarters, located at the corner of Broadway and Houston Street, is inaccessible to employees, they can’t take customer service calls.
“That’s one of the key lessons learned—what happens if the office loses power. We’re prepared for every eventuality except for that,” said Matthew Butlein, the president of the company. Mr. Butlein said the company will explore the possibility of duplicating the phone software at another location.
Companies located outside New York City have also been affected by the outage. MailChimp, an email marketing and list management company in Atlanta, has email and Web servers running in multiple locations around the country, including Dallas, New York and Seattle. Each location serves as a backup for the others, but is also the primary center for its region of the country.
According to Joe Uhl, who manages technology infrastructure at MailChimp, two of the three New York locations belonging to the company that hosts its servers failed because of flooding and loss of power. But his company’s equipment happened to be at the third location. “We dodged a bullet there,” said Mr. Uhl.
If it hadn’t dodged that bullet, some of the company’s customers wouldn’t have been able to access the Web portals they use to create new email campaigns, until those portals had been replicated at another center. “It wouldn’t have been seamless,” Mr. Uhl said, adding that the company is likely to explore improving its ability to do that type of replication.