Instability affecting ID CLOUD capabilities

Incident Report for unico skill

Postmortem

Postmortem: Authentication Flow Instability

Incident date: July 13, 2026

Impact duration: ~44 minutes

Summary

On July 13, 2026, we identified an availability degradation in the platform's authentication flows. The incident was caused by an incompatibility introduced during a rolling deployment of an internal component responsible for validating security tokens. The cause was diagnosed by our operations teams, and rolling back the update restored system stability within minutes.

Impact

The degradation lasted approximately 44 minutes and simultaneously affected multiple authentication flows on the platform, resulting in an error rate of approximately 0.9% in customer-facing services. During this period, some end users experienced instability and saw generic error messages (HTTP 500) when attempting to authenticate. Although the event lasted 44 minutes, it fluctuated between increased latency and unavailability.

Root Cause

The failure was caused by a software defect introduced in an update to the security token validation component, resulting in incompatibility between instances running different versions. This incompatibility was not detected in test environments, since it required the simultaneous, parallel execution of different versions. Additionally, the internal endpoint returned an HTTP success response even when validation failed, which prevented automated deployment monitoring systems from detecting the issue and automatically halting the rollout.

Resolution

Once the cause was identified, the on-call team executed a rollback of the affected component, immediately restoring availability of the authentication flows. The incident was detected by automated SLO alerts within minutes of the degradation starting, and escalation enabled full diagnosis and mitigation of the incident.

Lessons Learned

We will improve version-compatibility validation in rolling deployments, adjust the HTTP response behavior of internal endpoints to trigger automatic rollback of failing deployments, and standardize error handling to prevent failures from being masked before reaching the customer.

Commitment to Continuous Improvement

Incidents like this one reinforce our commitment to continuously improving platform reliability. We treat every incident as a learning opportunity, reviewing our processes, strengthening detection and prevention mechanisms, and continuously investing in more robust engineering practices. Our goal is to keep raising the bar on availability and transparency for all of our customers.

Unico Team.

Posted Jul 20, 2026 - 19:49 GMT-03:00

Resolved

Executive Summary and Impact

On July 13, 2026, we identified an instability in our identity verification capability, generating errors on a minimal portion of IDCloud requests, affecting multiple authentication flows. The issue was detected through automated availability alerts around 9:27 AM BRT, with a slight increase in errors already noticeable from approximately 8:00 AM BRT and a significant worsening starting around 9:07 AM BRT, reaching peak impact around 9:49 AM BRT.

Root Cause and Resolution

The root cause was identified as a gradual change applied to an internal authentication component, which caused a failure in session token signature validation. Once the root cause was confirmed, the engineering team reverted the change, and service availability returned to normal. The most critical phase of the impact lasted approximately 44 minutes.

Commitment and Next Steps

Engineering is evaluating improvements to the change-validation process for gradual rollouts affecting critical authentication components, including automatic rollback mechanisms in case of error spikes during this type of rollout. A detailed postmortem is expected to be shared internally as a next step.
Posted Jul 13, 2026 - 10:47 GMT-03:00

Monitoring

Dear Client,

Our engineering team has identified and implemented the actions to resolve the instability in the identity verification capability that was generating an error in a small portion of IDCloud requests.
At this time, the technical team is conducting detailed analysis in our data layer and continues to monitor the environment.

We reinforce that further updates will be sent shortly.
Posted Jul 13, 2026 - 10:06 GMT-03:00

Identified

Dear Client,

We have identified an instability in the identity verification capability, generating an error in a small portion of IDCloud requests.
Our engineering team is already mobilized for diagnosis and working to identify the root cause in order to normalize the service as soon as possible. At this time, the technical team is conducting detailed analysis in our data layer to mitigate the impact.

We reinforce that further updates will be sent shortly.
Posted Jul 13, 2026 - 09:30 GMT-03:00