(update 6th April 2088 – Customers who were affected who require the official service interruption notice, please contact support.)
If you’re a customer then you’ve seen this I hope. I sent it to all of our customer account contacts earlier this afternoon. Of course, you may have been affected and we’re not on the distribution list, or perhaps you unsubscribed from our announcements lists, so I thought I’d better post it where you may read it.
No excuses, we fell short of excellent today, and from my seat in the building, that’s not great. My email below says what I think we need to right now. We will be better. I promise.
(This email was originally sent on 29th March 2018 at approximately 16:00 BST.)
Network incident 29th March 2018
I would like to apologise to our customers that were affected by today’s outage. A full reason for outage will be issued once it has been supplied by our head office. This was made worse by customer inability to communicate with us by telephone, as our group phone system uses the same network.
Information is still being collated at this early stage, however, in short, the northern half of the Iomart network and therefore Glasgow, Edinburgh, Manchester and St Asaph datacentres and offices were affected by multiple fibre breaks across dark fibre routes on the Zayo network, so if your service is in any of these datacentre locations you would have been impacted.
We are acutely aware of the impact that even a moments interruption to service can have on customers whether they are using services just for themselves or whether they run a business.
We know you would like to know more and rest assured we will provide the aforementioned reason for outage as soon as it is supplied to us. We will also provide information as to what will be done to prevent such a reoccurrence in the future as soon as that is available.
At the time of writing services are being restored and most if not all customers should have service returned although there will be a recovery period during which route are reconverged and network routing returns to normal. Email that you’ve been waiting for will also start being redelivered one the next hour or so.
I know that some of our customers that have been affected by this may have already decided, or are contemplating moving their services as a result of this, and I’m reminded by something I was once told; “The good we do is rarely remembered but the bad we do is rarely forgotten”.
In 20 years of business Hosting UK has had outages, and we’ve always learned from them and built stronger and more resilient services that have ultimately benefitted our customers. Our uptime record I would argue, is good and I know from my own experience is better than many, but that does not mean we ever become complacent. We, as part of the Iomart group, *will* learn from this event.
It’s scant consolation, but I am deeply sorry that you’ve had this experience. You have my sincerest apologies.
Phil C Parry
Sales & Services Director