PRODUS - Integration Delays

Summary:

On 4th July, the Production East US2 environment encountered service disruptions caused by performance degradation in the storage subsystem. This resulted in delays in integration processing for a subset of customers. The issue was mitigated by decommissioning the affected nodes and migrating workloads to healthy ones. Services have since stabilised, and normal operations have resumed.

Customer Impact:

A subset of customers experienced delays in integration processing within the Production East US2 environment.

Root Cause:

A disruption in the underlying storage system impacted the timely movement of data volumes, leading to delays in integration task processing.

Remediations:

Affected nodes were replaced and configuration changes were applied to restore normal operations.

Future Mitigating Actions:

Enhance monitoring and alerting to enable earlier detection and response to similar platform issues.
We’ve applied targeted configuration changes to all environments to improve resiliency.

We have confirmed internally and with our customers that the Aera platform is now fully restored.

We appreciate your patience during this incident and apologise for any inconvenience that this issue may have caused. Our teams are now working on documenting a comprehensive root cause analysis which we will share with you shortly.

If you have any questions or experience any further problems please don’t hesitate to reach out to our Support team at Aera Support Portal

We have identified the cause of the reported issues. Our engineers have restored the services. We will continue to monitor to ensure no additional issues arise and will send a further update to confirm resolution.

You should now be able to resume normal activities however if you continue to experience any problems please contact our support team through the Aera Support Portal

Thank you for your patience and understanding whilst our engineers restored service.

Our engineers have identified the cause of the reported issues and are working to restore service as quickly as possible. We will continue to provide updates as restoration progresses.

Thank you for your patience and understanding whilst our engineers work to restore service.

Our engineers are continuing to investigate the delays affecting integrations that rely on data deduplication. We understand the business impact this issue may have and are working to restore service as quickly as possible. Again, we thank you for your continued patience and understanding.

We are continuing to work towards restoring service for the integration issues. Our engineers are diligently working to narrow down the root cause. We will continue to keep you informed as the investigation progresses. We appreciate your continued patience whilst we work towards resolution.

We are continuing to investigate the delays affecting integrations that rely on data deduplication. Our engineers are actively working to restore service as quickly as possible. Thank you for bearing with us whilst we work through these issues.

This notice is to advise you that we are receiving reports of our customers experiencing difficulties with the platform. We are actively investigating and will provide regular updates until the issues are resolved.

Our apologies for the inconvenience this may be causing and we appreciate your patience as we investigate further.