PRODUS- Cortex services are down

Major incident Production US Other Features
2023-05-31 06:39 UTC · 1 hour, 17 minutes

Updates

Post-mortem

Summary:
Following a recent upgrade on 30 May 2023, an issue was identified that resulted in a temporary interruption of Cortex services. The issue was escalated to the Infrastructure Engineering team where it was identified that unintentional loss of previous configuration changes during the upgrade resulted in network connectivity issues. To address the issue, necessary adjustments were made to ensure proper configuration and connectivity within our system was restored subsequently allowing full access of Cortex services to be reinstated. A gap within our change management processes has been identified and rectified to ensure we prevent any future occurrences. .

Customer Impact:
Cortex functionality was temporarily unavailable.

Root Cause:
Infrastructure Engineering identified that previous configuration changes were missed as part of the infrastructure upgrade leading to the disruption of Cortex services.

Remediations:
The required configuration settings were updated allowing full functionality to be restored.

Future Mitigating Actions:
Internal change management processes have been reviewed. A gap was identified within the process specific to these services and the process, and all stakeholder teams, have been updated to ensure any similar future occurrences are avoided.

June 12, 2023 · 10:41 UTC
Resolved

We have confirmed internally and with our customers that the Aera platform is now fully restored.

We appreciate your patience during this incident and apologise for any inconvenience that this issue may have caused. Our teams are now working on documenting a comprehensive root cause analysis which we will share with you shortly.

If you have any questions or experience any further problems please don’t hesitate to reach out to our Support team at support@aeratechnology.com

May 31, 2023 · 07:56 UTC
Investigating

We are continuing to investigate the issue with Cortex services. Our engineers are actively working to restore service as quickly as possible. Thank you for bearing with us whilst we work through these issues.

May 31, 2023 · 07:11 UTC
Issue

This notice is to advise you that we are receiving reports of our customers experiencing difficulties with the Cortex services in PRODUS environment. We are actively investigating and will provide regular updates until the issues are resolved.

Our apologies for the inconvenience this may be causing and we appreciate your patience as we investigate further.

May 31, 2023 · 06:39 UTC

← Back