“Plans are nothing; planning is everything” – Dwight D. Eisenhower
I was chatting with a couple of disaster recovery/business continuity folks yesterday over happy hour, and I listened as they talked about some of the challenges with disaster recovery planning.
One of the comments that stood out in my mind is that biggest challenge with disaster recovery plans is getting people to test them. Companies will pay a bunch of money for disaster recovery plan, and then let it sit on the shelf and collect dust.
I remember a team I worked on that was trying to build out a disaster recovery solution – basically just active/passive failover. Someone proposed that we switch which data center was active every few months. Everyone groaned at the idea.
Testing a disaster recovery plan might feel like a big waste of time because there are so many factors that won’t match the real disaster. From my experience testing, I think one of the most important factors in testing is designing the right test. You can spend a lot of energy “testing” something that really doesn’t matter.
So what are you testing for? Are you trying to see what happens when your servers fail? Or maybe that your alerting system works properly? Or that your team is trained to handle an external event, like an earthquake or fire? Do your backups restore without corruption? Depending on what you’re looking for, you need to design a test that exercises the process, and measures the important outcomes.
Just having a disaster plan is useless. Planning can help you think through your risks and what to prepare for. But testing the plan and putting your system and teams through the paces will help you improve resiliency and your teams capability to handle disasters.
“Blue Moon” testing is the worst way you can prepare for a disaster. Ensuring your teams test at least once a year, if not twice a year to validate they understand process and procedures in a disaster. More importantly, by testing – you uncover holes in your plan’s and your technology so you are able to address these before a disaster. Do not wait for the event to happen, even if your company is great at the cowboy style approach of recovery.
Thanks for the comment Kelly. Frequent testing is a good way to get the plan off the shelf!
Pingback: 5 Stages of Grief |
Pingback: 5 Disaster Preparedness Resolutions |