Cloud Fail

We have completed the work to address the issue. Error rates have returned to normal and all operations are now available.

There is a correction to our last note. The error code returned for CreateDomain and DeleteDomain requests is temporarily a 503. The cause of the elevated errors for put operations has been identified and corrective work is proceeding.

We are investigating elevated errors for PutAttributes and BatchPutAttributes requests for recently created domains. While we work to correct the issue, CreateDomain and DeleteDomain requests will temporarily return a 500 error code. Put operations for domains created before 9:34 AM PST are being processed normally.

CloudFiles will be performing routine maintenance this evening at 8:00 pm – 9:00 pm CDT. The system will remain available, however some customers may experience very brief connection lapses during this maintenance window. We will post a notification once maintenance is underway.

[UPDATE 8:00 PM CDT] Maintenance is now underway, we will update this post once maintenance has been completed.

[UPDATE 8:55 PM CDT] Maintenance has now been completed successfully.

The service has recovered and is operating normally.

We are suffering from an issue where GitHub’s API is timing out. This is affecting any deployments that uses GitHub integration.

We’re actively working on this problem however we don’t have an ETA at this time.

Thanks for your patience!

Update 9:30AM EST: It seems it was a hiccup on GitHub’s end. Integration is available again.

We have identified the root cause of the latencies and are currently working toward resolution.

Some Workers are still experiencing increased page load times. We are continuing to investigate.

Some Workers are experiencing elevated latencies with viewing available HITs. We are investigating.

Update: As of 3:57 am CST services have been restored to MySql50-06.wc1.dfw1.

The Rackspace Cloud system administrators are performing emergency maintenance on MySql50-06.wc1 as of 2:20 am CST.

During the maintenance  SQL queries utilizing
these clusters will be in a read locked state (INSERT, UPDATE, DELETE,
ALTER queries will be ignored) for the duration of the maintenance.

If
you have any questions, please contact a member of our support team via
live-chat or at the following telephone numbers: 24-hour toll free
1.877.934.0407 and INT +1.210.581.0407.