Building a DR site: Much more than a bunch of hardware
Recently I have gotten a lot of questions about building disaster recovery sites, and am actually working on a couple of DR designs. The one question I think I get the most is: “What is the most important part of building a disaster recovery site?” Every IT executive, manager, architect, or sys admin will give you a different response so I’ll just give you mine: Location
Now typically a site is picked for reasons other than disaster recovery. Eight out of ten times a site is picked based on location for business needs and IT gets a call “Hey Tom we’re opening a facility in New Orleans, Louisiana — we want to have that be a backup for this facility.” Well nobody stopped to think that wasn’t probably the wisest idea, but for some business reason it was — maybe because its close to a port? Whatever the reason, that was the situation you were dealt, but some IT executives are lucky. At larger companies the IT management is brought in to help make such decisions. SO if you’re one of the few lucky ones, what are things you need to take into consideration when picking a site?
As I said the most important is location. Most people think in order to have a DR site that means you have to actually have an operational presence in the area. In reality that isn’t the case at all. Since almost everything we do now days is based on technology you could easily have a facility managed by a small operations team in Boulder Colorado with IT operations farmed out to a local consulting firm. Sure there are some companies this would not work for, i.e.: a bank, but what about a distributor or e commerce company? Chances are if have a large enough web presence to worry about a DR site then you have multiple distribution centers throughout the country.
Assume for a second you’re a large distributor with operations all over the country. Perhaps you picked your distribution centers based on business needs for transportation or for your customers and not on disasters. Your corporate office is located in the heart of New Orleans Louisiana. Following the aftermath of Katrina, your distribution center in Mississippi, and Texas have been hammered by the storm. Corporate is completely in shambles with 5 feet of water in the computer room and virtually nothing left to the 1st floor of your building.
It’s OK though. You still have your third distribution center in South Carolina. The storm moved north in the east making communications to the east coast intermittent all day, but none of that matters because your small DR site is in Phoenix Arizona. The weather in Phoenix is 110, Sunny, and clear skies. Your website is still taking orders, your small operations team is answering the phones from concerned customers, and your backup outsourced operator standing by service is fielding overflow.
Why is this possible? Your IT staff took into account that building a DR site in South Carolina, Mississippi, or Texas was probably not the wisest choice they could make. They drafted a proposal that got rack space in a cage at the local data center in Phoenix. They installed a IBM blade center to host your website, front end exchange server, domain controller, and NetApp san syncing 24/7. They connected to this rack with a private Ethernet line running at 100mbs, and to the small local office location using the same. Your phone system is voip and AT&T knows your corporate office is down so its routing your 1800 number to the Phoenix DR site.
Congrats, you just took a 1,000+ person operation and ran it completely independently and free of just about every natural disaster with 25 people. This would allow the executive team to get to the DR site and begin rebuilding the operations more largely and asses the damage of their distribution centers.
So building a DR sites of today is something that is much more in-depth than simply buying hardware shoved into a room syncing your files. My feeling is a lot of engineers only look at the IT side of building a DR site — hence why I wrote this article. Consideration needs to be given to business needs, and cost limitations.
I could give you a check list for building your DR site, Needs, Location, Hardware, People, Connectivity, but that wasn’t the point I was trying to make here. There are cost effective ways to prepare for the worst and guarantee your future.




