CloudFail.net

Some users are experiencing no new updates to their timeline. We are aware of the issue and investigating.

Update (2/28 1:00p): We have pinpointed and resolved the issue.

As of 11:15 AM CST, the Rackspace Cloud system engineers have detected
an issue that’s causing degraded performance in our DFW Data center.  Our team is actively working to resolve the issue, any updates we have on this issue will be posted here.

If you have any further questions, please contact a member of our
support team by visiting us in live-chat or by calling 1.877.934.0407
or INTL +1.210.581.0407.

Update: As of 11:53 am CST, the Rackspace Cloud system engineers have resolved the issue and normal functionality has been returned to the DFW Datacenter.

[14:42 UTC] CloudFiles is currently performing emergency maintenance. During this period some JungleDisk customers will be in read only mode while the problem is investigated and corrected. We apologize for any inconvenience, and we will have an update for you soon.

[15:43 UTC] The issue has been discovered and we are currently working on correcting the issue. The maintenance should take approximately 4 hours and will conclude are 19:43 UTC. We will keep you up to date on our progress.

[17:51 UTC] The emergency maintenance has concluded, and all JungleDisk customers are currently read/write once more. Thank you for your patience.

UPDATE 3:25AM CST: The maintenance has been successfully completed and the network is currently stable. We do not expect any further interruptions. Your slice should now be fully visible and accessible in the SliceManager.

If you experience any issues, please join us in SliceChat.

UPDATE 12:56AM CST: The maintenance is progressing as expected, but is not yet complete at this time. Therefore, you may still experience intermittent packet loss to your slice as well as the issue of the slice not displaying correctly in the SliceManager. We will update you further when we have additional details to provide.

UPDATE 12:00AM CST: The maintenance window is beginning now. We will keep you updated with our progress. Please join us in SliceChat if you have any questions.

As some of you may know, we had several recent issues with the network in our STL-A datacenter. This maintenance will prevent those network issues from reoccurring. On February 27, from 06:00 to 10:00 UTC, we will be performing a maintenance to upgrade the network. During the maintenance window customers may experience intermittent packet loss and dropped connections as we make changes to the network. Slicehost engineers will be onsite, and we will keep the impact as minimal as possible.

Please note that during this maintenance, Slices in STL-A may not show up in the SliceManager. However, your slices have not gone anywhere and it is simply an issue of them not being displayed correctly. They will display correctly in the SliceManager when the maintenance has been completed.

If you have any questions please get in touch with our support by visiting SliceChat or via email at supp...@slicehost.com. Thank you for your patience and understanding.

DATE: February 27, 2010

TIME: 06:00-10:00 UTC

IMPACT: Intermittent network interruptions

The Rackspace Cloud’s system engineering team will be performing several upgrades to our WC2 infrastructure in our Dallas / Fort Worth, TX (DFW) data center to enhance stability and performance.  The maintenance is scheduled to last two (2) hours, beginning on the morning of March 6 at 12:00 AM CST and ending at 02:00 AM CST.  While connectivity during this window will be intermittent, some sites may be offline for as long as 90 minutes.  As always, we will do everything we can to keep the interruptions to an absolute minimum.



STORAGE ARRAY MAINTENANCE

During this maintenance window we will be performing a series of software upgrades and disk space additions to our storage array to increase capacity, stability, and scalability.  These upgrades will also allow us to further optimize our storage array’s performance, equating to faster data load times for your applications and better stability.  During the maintenance period you may be unable to access, upload, or make changes to your website content.   We will certainly do everything possible to limit the amount of time your account will be affected.



We apologize for the short notice of this maintenance.  It is critical, however, that the work be completed in as timely of a manner as possible to ensure increased stability and scalability.  This maintenance activity is a direct result of our in-depth infrastructure review that we have completed on our WC2 cluster.

Update:  As of 1155pm CST, Rackspace Engineers will be starting the maintenance for this evening.  We will continue to update the post as the maintenance progresses. 

Update: As of 1:21AM CST, Rackspace Engineers have completed maintenance for this evening.  If you have any questions, please contact a member of our
support team by visiting us in live-chat or by calling 1.877.934.0407
or INTL +1.210.581.0407.

As of 7:35 AM CST, the Rackspace Cloud system engineers have detected an issue that’s causing degraded performance on our ASP1-5.WC2 cluster in our Dallas / Fort Worth (DFW) data center.  This is an isolated issue impacting this single node only.  We will update this post as the status changes.

UPDATE: As of 8:30 AM CST, system engineers continue to address the performance issues affecting ASP1-5.  We will post here with updates as soon as they are available.

UPDATE: As of 9:50 AM CST, Rackspace Cloud Engineers are continuing to work on the issues affecting our ASP1-5 cluster.  As of this update, the issues are still isolated to this cluster. 

UPDATE: As of 10:30 AM CST, the team remains fully engaged in correcting this issue.  Some sites in our ASP1-3 cluster may also be affected at this time.  We will continue to keep you updated via this post.  

UPDATE: As of 11:00 AM CST, the team has made progress, but they are continuing to actively address the issue.  The performance degradation affecting ASP1-3 has significantly improved and sites should be loading properly.  We will continue to address the ASP1-5 cluster.

UPDATE: As of 11:10 AM CST, at this time, all sites hosted on ASP1-5 should now be loading properly.  The team has continued to monitor the cluster over the past half-an-hour and site performance has remained consistent.  Once again, we appreciate your patience as we corrected this issue.

As of 2:25 AM Rackspace Cloud engineers identified and issue with our Mysql 50-50 database cluster and have placed it in Read Only Mode currently.

If you have any questions or concerns please contact our support via
live chat or at 1-877-934-0407 international +1.210.581.0407.

Update: As of 4:55 AM CST, engineers have corrected the issue
affecting MySQL 50-50 in SAT.  If you are experiencing any further issues,
please
contact a member of our support staff.

Thank you.