The support we offer our contract customers does not stop at 5pm on a Friday.
Last Friday evening I took a call from a client who was reporting that the power to his office had been going on and off all afternoon and now the power company (who were digging up the street outside) had told him there would be major power outages all weekend. This of course concerned us as it could have an effect on several systems they rely on – servers, email, phones and backups.
It’s not unusual for me to take a call in the evening from a client and then sort a problem for them but this time I was camping in the lake District – see here.
Straight away Martin spoke to the power company to get some idea of the scale of the problem and double checked the status of the current offsite backups from the night before.
The most important thing for the client was to have someone from Octagon available over the weekend to manage the situation. So from the Windermere site we organised:
- Early on Saturday, Martin to check by remote whether any of the systems were working at the office – and to report to Diana.
- Diana to be onsite when the office opened on Saturday morning – the power was however still off at that point.
- Martin to go onsite Saturday afternoon, when there was power to restart the servers, and check they were functioning correctly. Whilst he was there, the power company informed him that the power would be going off again for an extended period, so he and I took the decision to close all the servers down and stop any possibility of damage to the system.
This was the view I had when Martin and I decided to turn the servers off – see more photos here.
On Sunday, whilst stopped for coffee at a service station, the client phoned again to tell me that the power was now back on, so I stopped off in Lincoln city centre before going home to turn everything back on and then ran our systems check list to make sure it was all working. The last job for Sunday was to let the client know that everything was ready for Monday morning.
At 6am on Monday, I logged into the system using my iPhone and ran through our checklist – just to make sure all was well.
At 8.30am the client opened his office as though nothing had happened!
Another crisis solved.
