Hello.

We would like to apologize for the inconvenience caused by the interruption to your services on UK Data Center the morning of November 15th and to explain the sequence of events, based on the information currently available to us. 

Reading experienced a regional power failure which took the two power feeds out from two separate substations at the same time.  Ultimately, this regional power failure resulted in problems transitioning from our dual mains supply to backup generator power and as a result not all of the Data Centre successfully transferred to backup power.   

This failure to transfer to generator power meant that some of the Data Centre continued on UPS supply and services remained operational.  Ultimately the generators were not able to transfer the power load and the timeline of events is as follows: 

08:12:02 - Primary and secondary high voltage feeds from the grid fail

08:12:17 - Start command to emergency diesel generators is automatically sent

08:12:56 - Generators have started, synchronised with each other, and providing power to site

08:13:31 - Generators have excitation failure as load is stepped on, due to reasons to be fully investigated.

UPS continued to feed the Data Centre and you will have experienced issued from approximately 08:39 GMT. 

The backup generators designed to carry double the total Data Centre power requirement and are tested for load on a monthly basis.  Data Center have a series of engineers on site currently investigating why there was a failure in the process of generators accepting the power load and this issue is being resolved as a priority.  The engineers have focused on settings in the synchronizing controls and a fix has been implemented.  The process of testing this fix is currently being coordinated and thorough testing will be undertaken. 

We will be receiving a full ‘Reason For Outage’ report from the data center engineers and this will provide greater detail. 

Mains power was restored to the site around 10:15 GMT and the data center engineers immediately began the process of restoring full functionality across all devices.  Unfortunately for some Servers this process required manual intervention and as a result you have experienced a delay in your services coming online.  Please accept our apologies for this delay. 

We appreciate your patience and cooperation during this period.

Best Regards,

SkyNetHosting.Net Inc.