Disaster recovery: Ready or not?

When information and communication systems are disrupted, it’s much more than an inconvenience.

Digital records and legal documents can be lost, employee and customer trust can be weakened, and productivity and revenue can be threatened severely.

As we have seen during events such as natural disasters, situations that shut down business-critical systems and applications for any length of time (or wipe them out completely) can have devastating direct and indirect costs to the business — costs that make it absolutely vital to have a solid disaster recovery (DR) plan.

But even in the wake of some of the most severe disasters on record, it appears that many CIOs aren’t preparing their companies for the next one.

Sluggish response

What explains this lack of preparation and responsiveness? For one thing, companies have to be strategic about how they allocate resources during recovery — you’ll be transferring and uploading loads of data, and bandwidth can disappear very quickly. It’s crucial for IT to prioritize customer-facing and other business-critical systems, while certain backed-up data (such as email archives) can be put off until later.

When establishing priorities for DR, business continuity also needs to be understood. Sometimes, even while certain IT systems are down, the business can still operate via manual or alternative processes for reasonable periods of time. When that’s the case, other systems — ones that can’t operate in any alternative form — should take priority.

As it turns out, though, what’s really holding companies back isn’t how and where they’re backing up their systems and data — it’s failures in policy and procedure.

DR must be decisive

What IT has to know about declaring disasters is that it doesn’t really matter what kind of incident is affecting your business. It might be something climactic like a hurricane, but more often it’s a mundane incident like a power outage and simple human error. What makes it a disaster is the extent of the impact on your business. So to determine how to respond, IT needs to ask, “How long will it take to restore the systems and/or data affected by this incident?” Only a portion of your systems may be compromised, but if a full deployment of your DR plan would take 24 hours — and addressing the individually compromised parts of your business would take just as long — it might be wise to declare a disaster and go ahead with the full DR execution.

This is why careful, ongoing monitoring is so critical. You have to know what is affected and how long it’ll take to restore — a process that begins, actually, well before disaster ever strikes. You need to perform a thorough assessment of what optimal performance looks like under normal circumstances, and only then can you judge the damage in an emergency and estimate what it’ll take to recover.

As for the lack of communication between recovery staff, I have to say, it’s disappointing to see this listed as a top challenge. I have written multiple posts about why and how to draft a complete and rigorous incident response plan. With numerous stakeholders and compliance issues to satisfy during recovery, communication breakdowns simply can’t be allowed to occur.

And just like with an incident response plan, testing your DR plan is critical. You want to identify issues and gaps during a test and not during a real disaster. The pressure will be high enough during a real event, and you want to be certain roles and responsibilities are clear and that no underlying technical issues exist.

The final challenge — mismatched expectations — is a tricky one. Business leaders, understandably, want recovery to happen now. But IT is being stretched increasingly thinner, with responsibilities and expectations outpacing budgets. If IT finds it hard to muster the preparation and capabilities demanded by executives — not to mention customers, who are equally interested in a fast recovery — then outsourcing is worth considering. An experienced vendor not only frees up in-house staff to focus on more strategic agendas, but provides invaluable insight and resources toward creating a resilient infrastructure.

Indeed, anything IT can do to prepare for the worst is worth doing. Unexpected weather and common mishaps will happen — whether you’re ready or not.