Cloudflare errors across Spektrix system
Incident Report for Spektrix
Resolved
We have continued monitoring this, and have not seen a recurrence of these Cloudflare errors since 12:15 BST / 07:15 EDT.

These problems started around 4pm BST / 11am EDT yesterday, 18 September. We initially understood Cloudflare to have identified the problem and implemented a fix, but they identified an additional issue around 10:30 BST / 05:30 EDT this morning, which has now been resolved.

Thank you for your understanding and patience whilst we we worked to resolve this. If you are still experiencing any specific issues, please reach out to our Support teams on support@spektrix.com or on +44 (0)20 7183 3586 (UK) or +1 (646) 569-9097 (North America)
Posted Sep 19, 2023 - 13:00 BST
Monitoring
We have been closely monitoring the frequency of errors following the fix implemented by Cloudflare, and we are confident that this has significantly reduced the frequency of error screens that you and your customers may have been seeing. We have also seen API requests complete without error and websites be updated successfully.

Thank you for your patience throughout this.
Posted Sep 19, 2023 - 11:39 BST
Update
Cloudflare have been able to identify a further problem which we believe is the cause of the issues that you have been seeing this morning. They are implementing a fix, and we have already seen a large reduction in the number of errors.
Posted Sep 19, 2023 - 11:04 BST
Update
We've investigated further this morning, and we believe that clients in Scotland may be adversely affected, due to how their web traffic is being routed. If you are able to alter your routing (e.g. using 4G or a VPN), you may see your connection become more reliable.

We have also seen some issues with web integrations displaying incorrect dates or wrong availability. Some API requests are occasionally getting error responses at the moment.Please loop in your web developer who should be able to force another API request in order to retrieve the correct data.

We are continuing to work with Cloudflare to escalate this issue, and we thank you for your patience and understanding.
Posted Sep 19, 2023 - 10:26 BST
Update
Cloudflare, one of our infrastructure providers, are still experiencing problems this morning, and so users may still see error pages sporadically throughout the Spektrix interfaces, and customers may occasionally be unable to load iframes. Refreshing the page should resolve this.

These problems started around 4pm BST / 11am EDT yesterday, 18 September. We initially understood Cloudflare to have identified the problem and implemented a fix, but as we're still seeing reports of this error, we are continuing to reach out to them on this.
Posted Sep 19, 2023 - 08:35 BST
Update
Update: We are aware that some users are still having issues connecting to Spektrix, and may be experiencing cloudflare error pages. This may show as 'Error 1016 - Origin DNS error'.

We are continuing to investigate this and will update you as soon as we are able.
Posted Sep 18, 2023 - 22:02 BST
Identified
Our investigations have identified that this is related to a problem that Cloudflare are experiencing. Cloudflare are one of our infrastructure providers, which is why some users are experiencing these error pages.

We have spoken to Cloudflare, who have been able to identify the issue and they are currently implementing a fix. In the meantime, these errors usually resolve upon a refresh of your page.
Posted Sep 18, 2023 - 17:29 BST
Update
We are continuing to investigate the cause of some users having difficulties accessing the Spektrix system. We are finding that these are sporadic errors, that usually resolve upon a refresh of your page.

We are continuing to investigate this issue and will provide updates as we have them.
Posted Sep 18, 2023 - 16:55 BST
Investigating
We are aware that some users are having issues connecting to Spektrix, and may be experiencing cloudflare error pages. This may show as 'Error 1016 - Origin DNS error'.

We are currently investigating this and will update you as soon as we are able.
Posted Sep 18, 2023 - 16:30 BST
This incident affected: Spektrix System (UK and Ireland, US and Canada).