CloudFail.net

Please be advised our Chicago facility has a scheduled maintenance from Friday, September 3rd, 2010 at 08:00 UTC to 09:00 UTC. We will be re-routing our Chicago facility at 05:00 UTC to prepare for this maintenance. All services will be automatically re-routed to the closest alternate facility and no services will be impacted by this maintenance.

We’re currently receiving some abnormal traffic patterns at our Seattle location. To avoid any potential interruption of services, we’ve withdrawn our resolver anycast announcement from the site. All users will automatically be re-routed to the closest alternate site.

Additionally, users routed to our Seattle or Los Angeles location may have experienced typo-correction services being temporarily disabled from  Thu Aug 26 10:00 UTC to Thu Aug 26: 21:00 UTC. All services have been re-enabled at both locations.

UPDATE: Seattle has been re-enabled and is now serving DNS traffic again.

We’re currently receiving abnormal traffic patterns at our Seattle location, which may have a negative impact for users on certain carriers. To avoid any potential interruption in service, we’ve re-routed the site to alternate facilities. Users normally destined to our Seattle location will be routed to other facilities.

UPDATE: Traffic to seattle has been restored.

We’re currently receiving abnormal traffic patterns at our Seattle location, which may have a negative impact for users on certain carriers. To avoid any potential interruption in service, we’ve re-routed the site to alternate facilities. Users normally destined to our Seattle location will be routed to other facilities.

UPDATE:  Traffic to seattle has been restored.

This is a continuation update for work still being performed from Friday night.We will be performing maintenance at our London site starting Friday, July 23rd at 21:00 UTC.  Work is expected to be completed by 01:00 UTC the following day. All services will be re-routed to alternate facilities and no services will be impacted by this maintenance.

UPDATE: Due to some delays we are extending the maintenance window by 1 hour.  Expect services in London to be restored by 02:00 UTC.

UPDATE: The window is now being extended to 5:00 UTC.

UPDATE: Maintenance is complete and service has been restored to London.

We will be performing maintenance at our London site starting Friday, July 23rd at 21:00 UTC.  Work is expected to be completed by 01:00 UTC the following day. All services will be re-routed to alternate facilities and no services will be impacted by this maintenance.

UPDATE: Due to some delays we are extending the maintenance window by 1 hour.  Expect services in London to be restored by 02:00 UTC.

We’ve experienced a power failure in our London facility. We’re not sure yet if it’s limited to our rack, cage, room or the building. All traffic has failed over to Amsterdam. We will update you when we have more information.UPDATE: Power has been restored to our London facility and is back in operation.

We’ll be performing routine maintenance at our London site between 21:00 and 22:00 UTC on Wednesday the 14th of July. All services will be re-routed to alternate facilities and no services will be impacted by this maintenance. We’re re-routing traffic to alternate sites as a precautionary measure only.

UPDATE: We’ve brought our London facility back online and finished this maintenance well ahead of schedule. No services were impacted by this maintenance.

Our Miami facility is currently offline. We’re working with the facility to determine the nature of the issue. All services at this site are being routed to alternate sites.

UPDATE: The facility is back online. This appears to have been a routing issue with one of our carriers which is now resolved. We’re working with them to determine what the cause was. No services should have been impacted by this.

We had a couple issues that occurred Thursday June 10 between 7:00 UTC and 14:00 UTC.  Both of these issues took longer to resolve than our standard response time due to a obscure bug in our monitoring systems.   We’ve identified the root cause of the monitoring issue and we’ve taken a number of steps to ensure it cannot happen again. In addition, we’ve added a significant amount of internal and external monitoring to help isolate and identify issues with any of our services. Our utmost apologies for any inconvenience generated as a result of these issues.

The Amsterdam issue was with the navigation proxy at our Amsterdam site intermittently from 10:25 UTC to 13:00 UTC.  We’ve disabled the proxy in that location pending an on-going investigation to identify the root cause of the issue. During that time frame, the proxy was experiencing very bizarre traffic patterns and we’re doing extensive testing to ensure our systems can handle similar traffic patterns in the future.