Aera Platform is unavailable

Summary:

On the 5th Sept 2024, internal monitoring alerted the Production Services Engineering (PSE) team that the Production US (ProdUS) environment was unavailable, leading to users experiencing disruption while attempting to access the Aera Application. Upon investigation, it was discovered that a specific complex transaction type was not being executed as expected, resulting in the access disruption. To restore service, the impacting transactions were terminated and service subsequently resumed.

Customer Impact:

The ProdUS platform was unavailable, causing disruptions to access Aera Application.

Root Cause:

The issue occurred due to resource contention among the backend services as a result of a complex transaction generating unexpected results.

Remediations:

To address the issue promptly, our team terminated the impacting transactions, restoring normal functionality.

Future Mitigating Actions:

Monitoring and alerting for the impacted backend service has been reviewed; a gap was identified as well as enhancements to existing alerts. The PSE team has implemented the enhancements and the additional alerting will be deployed shortly.
Internal change management processes have undergone review. Additional steps required within the process were identified, and necessary updates have been made and communicated to all stakeholder teams.

We have confirmed internally and with our customers that the Aera platform is now fully restored.

We appreciate your patience during this incident and apologise for any inconvenience that this issue may have caused. Our teams are now working on documenting a comprehensive root cause analysis which we will share with you shortly.

If you have any questions or experience any further problems please don’t hesitate to reach out to our Support team at support@aeratechnology.com

We have identified the cause of the reported issues. Our engineers have restored the Aera platform and service has been restored. We will continue to monitor to ensure no additional issues arise and will send a further update to confirm resolution.

You should now be able to resume normal activities however if you continue to experience any problems please contact our support team support@aeratechnology.com

Thank you for your patience and understanding whilst our engineers restored service.

Our monitoring tools are reporting increased error rates while accessing the platform. We are actively investigating and will provide regular updates until the issues are resolved.

Our apologies for the inconvenience this may be causing and we appreciate your patience as we investigate further.