Disaster Recovery planning can be painstaking. There are so many nuanced areas of focus that it is easy to miss key information that could hinder or block restoring systems and data within the time frames required by the organization. Exercising plans is essential to help illuminate these hidden risks. Here are some items we frequently find missing even in very mature disaster recovery plans.
1. Escalation Criteria/Requirements – ensure the plan identifies a clear procedure for escalating not only the detection of an issue that may require plan activation, but the procedure for notifying key contacts when the recovery is not going according to plan. Contact information is essential, of course, but identifiable and measurable criteria that, if met, would require the notification of key staff members is often undocumented. Without these guidelines in place, key performers will continue to bang their heads against a wall while the clock ticks away when a simple report on the road block or request for assistance could have easily saved valuable time.
2. Data Backup – Few IT professionals overlook data backup under normal circumstances. That isn’t always the case when disaster recovery environments are being utilized. Ensure that the plan contains instructions for enabling the backup of data being entered into DR systems. The business users of the backup systems should also be alerted as to the RPO for the DR environment. If the RPO is not socialized, the assumption will be that the DR systems have the same capabilities as production, and any loss of data in the event of a DR system failure would make the post-incident review more than uncomfortable.
3. Special Authorities – document the special access rights necessary to perform recovery tasks. Do not assume that personnel with access will be available. Capture the procedure for obtaining the IDs/passwords necessary in the event that key performers are not able to work.
4. Log of Actions/Events – capture a log of the actions taken during the recovery. It’s unfair for management to assume that every decision made during the event will prove to be the right choice. It’s not unfair to assume that a decision made was the right one based on the situation at the time when the decision was needed. The ability to refer to a comprehensive log of actions and events will prove handy in responding to questions when reviewing the incident. The log will also be useful as a means of improving recovery plans.
5. Failback Procedures – ensure that the plan contains the procedures to reverse any automatic or manual failover performed during the recovery. DR plans are often remiss in detailing how to return to normal. The process may not be as simple as a stepping back through the failover procedure. Make sure the procedure is exercised and well documented.
