Service is operating normally: [RESOLVED] Elevated error rates

Between 2:10 PM PDT and 4:15 PM PDT today, a small number of queues experienced elevated error rates and latencies. Our monitoring correctly alerted us of this issue and we took steps to mitigate impact to customers. The underlying issue was caused by a recent deployment to a small number of hosts. This deployment, along with an unusual traffic pattern this afternoon, triggered an edge case that resulted in the increased error rates. We have since rolled back this deployment and are in the process of updating our test procedures to take this traffic pattern into account for future deployments. The service is now operating normally.


Comments are closed.

Get Adobe Flash playerPlugin by wpburn.com wordpress themes