Last week’s Melbourne Trains debacle provided graphic evidence that organisations need to be constantly alert and prepared for the worst, when it comes to their ICT operations!
Whilst tens of thousands of commuters were left to fume about the inadequacy of their Train systems, Uber was the only winner, hiking fares from a normal $8.00 to $40.00 plus for a local trip, in a typical display of concern for customer!
Melbourne Trains claim to have traced the computer glitch to “a single server” in the Metro Control Room! (http://www.9news.com.au/national/2017/07/14/03/34/possible-compo-after-vic-rail-meltdown) They offered cold comfort in stating that they were confident the system was not hacked nor subject to cyber attack.
Briefly mentioned in the reporting of this peak hour meltdown was the fact that the back -up systems did not work.
Herein lies the real problem, and a salutary lesson to everyone involved in the ongoing operation of their organisations’ IT Services.
There are 4 important ingredients in preventing your own “melt down”:-
1. Prevention (as much as humanly possible) of a major outage by having a sound and robust IT Strategy and architecture to provide the resilience required to support business operations. Consider the resilience of ICT services during design, procurement and implementation to meet or exceed business and customer expectations.
2. Recovery. Develop a sound and robust Disaster Recovery Plan that addresses the entire organisation and train your team in business expectations and their roles and responsibilities.
3. Action. As a crucial part of the Disaster Recovery Plan, have a well- planned and tested Back Up Plan. The key word here is TESTED! It is cool to have a beautifully documented Disaster Recovery Plan and a Back Up Plan – but it is a good idea to test the process regularly!
4. Update. Implementation the opportunities for improvement identified during testing will only improve your service resilience, staff knowledge and disaster recovery capability.
Big BTW here – if you use 3rd party Cloud Services or Co-locate in a Data Centre – this still applies to you! You need to clearly understand what Disaster Recovery and Back Up Plans your Supplier has and if these are aligned to your business needs. If you don’t understand – get someone involved who does know what the correct processes should be, in the event of a major outage of some kind.