UATIRL- Streams service is down

Summary:

On 10th July 2023, it was reported that the streams executed were failing and there was no error in the monitoring console.
Upon investigation, it was found out that due to a large number of executions of objects there was a sudden increase in the number of logs generated, which impacted the backend service resulting in failure of streams.

The engineering team identified and fixed the issue and deployed the fix to the UAT environment.

Customer Impact:

The stream execution was failing without any error in the monitoring console.

Root Cause:

A sudden increase in the number of logs generated resulted in unavailability of the backend service.

Remediations:

The engineering team identified the software fix and deployed it to the UAT environment.

Future Mitigating Actions:

The missing test case has been added to the test suite.

We have confirmed internally and with our customers that the Aera platform is now fully restored.

We appreciate your patience during this incident and apologise for any inconvenience that this issue may have caused. Our teams are now working on documenting a comprehensive root cause analysis which we will share with you shortly.

If you have any questions or experience any further problems please don’t hesitate to reach out to our Support team at support@aeratechnology.com

We have identified the cause of the reported issues. Our engineers have restarted the service and streams has been restored. We will continue to monitor to ensure no additional issues arise and will send a further update to confirm the resolution.

You should now be able to resume normal activities however if you continue to experience any problems please contact our support team support@aeratechnology.com

Thank you for your patience and understanding whilst our engineers restored service.

We are continuing to investigate the issue with the Streams. Our engineers are actively working to restore service as quickly as possible. Thank you for bearing with us whilst we work through these issues.

This notice is to advise you that we are receiving reports of our customers experiencing difficulties with the platform. We are actively investigating and will provide regular updates until the issues are resolved.

Our apologies for the inconvenience this may be causing and we appreciate your patience as we investigate further.