Salesbuildr - Notice history

Earlier today customers in Europe experienced an outage.

The issue was caused by a software defect introduced in a recent update, which led to a critical system component running out of capacity. We resolved the issue and services were restored by approximately 05:50 CEST.

We apologize for the disruption. We are putting additional safeguards in place to prevent a recurrence.

EU web application is now operational! This update was created by an automated monitoring service.

Earlier today customers in the United States and Australia may have experienced errors when using distributor-related features (for example searching or retrieving distributor pricing and availability).

This was caused by a related outage in our European environment that affected a shared Distributor Hub service. Core Salesbuildr functionality in the US and Australia was otherwise unaffected.

We apologize for the disruption.

APAC web application is now operational! This update was created by an automated monitoring service.

A brief connectivity disruption during a deployment caused EU servers to cycle for ~20 minutes; other regions were unaffected. We're adding resilience and monitoring improvements to prevent recurrence.

EU web application is now operational! This update was created by an automated monitoring service.

EU web application is now operational! This update was created by an automated monitoring service.

EU web application is now operational! This update was created by an automated monitoring service.

EU web application is now operational! This update was created by an automated monitoring service.

EU web application is now operational! This update was created by an automated monitoring service.

EU web application is now operational! This update was created by an automated monitoring service.

EU web application is now operational! This update was created by an automated monitoring service.

This incident has been resolved (but we'll keep actively monitoring)

The Elasticsearch cluster backing distributor product search has been restored to a healthy state. Search response times and indexing latency are back to normal. We have applied configuration changes to the index to prevent recurrence of this specific failure mode and are following up with engineering work to make this class of incident self-healing in the future.

The root cause has been identified as a search index that fell behind on background maintenance, leading to high resource pressure on our Elasticsearch cluster. We are applying tuning changes to the index and clearing the backlog. Search results may continue to be slow during recovery.

EU web application is now operational! This update was created by an automated monitoring service.