CloudFail.net

Dear customers. We are experiencing some slowness on some Atlanta VPSes, including all Cloud Hosting servers located on Atlanta cloud. Our L3 team is working on this problem. We apologise for the inconvenience.

[Update]
Feb 04 20:22 – We had to start the file system integrity check on one of our Cloud Hosting servers. This process is going to take about 2 hours.
Feb 04 22:07 – Due to some problems appeared with the new disks, we are going to move all our Cloud Hosting servers to the old disks. The servers will be rebooted. We apologize for the downtime caused by the reboots.
Feb 04 22:38 – Email sent to customers:

Dear Customers,
As you may be aware there was a failure on the cloud hosting platform earlier today. This appears to be linked to a filesystem issue on the cloud storage. As such we have taken the decision to roll back to a backup of the cloud hosting platform that is 2 days old to allow sites to be restored.

** We aim to restore the full cloud platform, with current data, within 12-24 hours **

As a temporary measure we have blocked incoming email to avoid any new emails getting lost. Any pending emails will be stored temporarily on backup email servers. Once the cloud platform is fully restored we will allow these emails to flow into the restored service. Whilst we do understand the importance of email we have taken this decision to make sure no email is lost or left out of sync. All other functions (website/control panels) are up and running.

We apologise for this outage, and are working as quickly as possible to restore service.

Once this is restored we will be sending a full RFO (Reason for Outage) to customers, including any credits that are due under the SLA.

Should you have any questions about this please don’t hesitate to contact us on supp...@vps.net.

–

We’ve got Backpack serving up files happily again. We are sorry for the prolonged nature of this issue. It took us an unusually long time to debug this one despite our best efforts.

Dear customers. Our backup SAN on Amsterdam cloud is unreachable. We strongly recommend you do not try to restore from backups or create new backups until issue will be resolved. We apologise for the inconvenience.

We’re working on an issue preventing files upload to Backpack from downloading correctly.

07:00 GMT – 15:00 GMT – We are going to update OnApp software on these clouds. Power action buttons might be temporarily disabled within this time.
This is done. Thank you for your patience.

Due to an issue with our CloudFiles Log Delivery system, users who have public CDN containers with Log Delivery enabled will not see logs delivered.

Our developers are working on a solution to be put in place. All accumulated CDN logs will be eventually delivered once our Log Delivery Service is re-enabled.

[3pm MST] We are having maintenance on our Atlanta Cloud Hosting servers. All the data is currently moving to more stable SANs.

[4:06pm MST] Everything should be up and back to normal. We are sorry for any inconveniences caused.

Thank you for your patience, and business!

Dear Customers;

We are experiencing an issue with SAN 4. Our L3 team is working on this. We apologise for the inconvenience.
Feb 02, 11:30 AM GMT SAN issue has been solved.

All affected VMs have been started and should be online shortly. Should you have any questions feel free to contact our Support Team.