System Slowness

Incident Report for Spektrix

Postmortem

What happened?

At 10:00 (GMT) on Monday 28th October, we identified slowness within the Spektrix system. Until approximately 13:40 the system remained slower than usual, and was inaccessible for brief periods in the morning. This affected both system users and customers booking online.

We immediately began work to resolve the problem at 10:00 when we identified that our load balancer, a component which distributes incoming requests to our servers, was not performing as we would expect.
After carrying out our standard tests and procedures on the load balancer, system speeds began improving from around 10:40. Work to optimise system performance continued.
System speeds got progressively better throughout the morning and early afternoon and were back to normal levels by 13:40.

We continued to closely monitor system performance for the rest of the afternoon; we have not seen any slowness since the problem was resolved on Monday.

‌

We are committed to providing the best possible service for you and your customers, and are sorry for the disruption you experienced yesterday.

If you have any questions, please get in touch with support@spektrix.com.

Posted Oct 29, 2019 - 10:56 GMT

Resolved

Earlier speed issues have been resolved and all systems are running normally.

Posted Oct 28, 2019 - 14:56 GMT

Update

We've identified the issue and our engineers believe speeds should now be back to normal. We'll continue to monitor the situation. Thanks again for your patience.

Posted Oct 28, 2019 - 14:23 GMT

Monitoring

System speeds look to be returning to normal after work has been completed by our engineers. We are continuing to monitor the speeds and will keep you updated. Thanks again for your patience!

Posted Oct 28, 2019 - 13:48 GMT

Update

You and your customers online may experience a period of slow clicks shortly whilst our engineers carry out work. We are aware of this and will continue to monitor throughout. Thanks again for your continued patience.

Posted Oct 28, 2019 - 13:08 GMT

Update

#amber Thanks again for your patience, we're continuing to monitor the system and will keep you updated! As a reminder, if your Spektrix App has disconnected, restarting this should get you back up and running.

Posted Oct 28, 2019 - 12:27 GMT

Update

We're continuing to monitor the system's speed now we've identified the cause of these problems. If your Spektrix App went offline, restarting it should fix this. Thanks for bearing with us!

Posted Oct 28, 2019 - 11:54 GMT

Identified

We've found the cause of this morning's slowness and are starting to see some improvements in speed. We'll continue to keep you updated here. Thank you for your patience this morning.

Posted Oct 28, 2019 - 11:30 GMT

Update

We're continuing to investigate slow speeds on the system but we'll keep you updated here. Thanks for bearing with us.

Posted Oct 28, 2019 - 11:06 GMT

Update

We're aware system slowness is still a problem; our engineers' investigations are continuing and they're working to get the system back up to speed as soon as possible.

Posted Oct 28, 2019 - 10:51 GMT

Update

Spektrix is now back up and running, but we're still experiencing slowness. We'll update you here as our investigation continues. Thanks for being patient with us.

Posted Oct 28, 2019 - 10:39 GMT

Update

We're aware that Spektrix is currently inaccessible. We're working to fix the problem and will let you know as soon as we've made progress here.

Posted Oct 28, 2019 - 10:27 GMT

Update

We're continuing to investigate these slow system speeds; thanks for bearing with us as we do. We'll keep you updated here as soon as we have more news.

Posted Oct 28, 2019 - 10:21 GMT

Investigating

We are receiving reports of slowness across the system. We are currently investigating the causes of this and are working hard to fix this. We will update shortly with more news when we have it.

Posted Oct 28, 2019 - 10:11 GMT

This incident affected: Spektrix System (UK and Ireland).