Metabase Security Incident<br>On 3 August 2026, an attacker exploited a zero-day vulnerability in Metabase, the third-party analytics tool we use internally, and gained read access to a database holding a copy of Checkly operational data. The attacker bypassed authentication and obtained an administrator session on our Metabase Cloud instance. Metabase has since blocked the attack, patched the vulnerability, and published a security update. The attacker did not access our production platform.<br>If you put authentication values directly in your check configurations, or you use OTEL API keys for trace collection, or you want to be extra safe about potential data being accessed in an encrypted state, please rotate your credentials now.<br>The rest of this post explains what was exposed, what was not, and what we are doing about it.<br>We are sorry this happened. The vulnerability was in a vendor's product, but protecting your data is our job, and this incident put some of it at risk.<br>What happened<br>Metabase is a business intelligence tool we use to analyze product usage and operational metrics. It connects to a data warehouse that holds a copy of Checkly operational data, separate from our production platform.<br>The zero-day allowed the attacker to create an administrator session without valid credentials. No password was stolen, and no Checkly system was directly breached. With that session, the attacker queried the connected data warehouse through Metabase for about 26 minutes on August 3rd, 2026. All activity was read-only.<br>Metabase discovered the attack against Metabase Cloud the same day, blocked the endpoints the attacker used, and patched the vulnerability. They notified customers, law enforcement, and engaged a third-party forensics firm on August 6th. On August 10th, we concluded our investigation of the attacker's activity in our instance using detailed logs Metabase provided to complete our own analysis of the affected systems.<br>What may have been exposed<br>The data warehouse contained check configuration and account data, potentially including authorization keys or headers if your checks send them. Specifically:<br>Credentials you store as Checkly secrets are encrypted at rest and injected only at run time. These were not exposed in plaintext .<br>Checkly stores custom values added directly in a check's configuration (custom headers, cookies, query string parameters, or authorization headers) in plaintext. You should consider these values exposed.<br>Cryptographic hashes of OTEL API keys used for trace collection were also accessed.<br>What was not affected<br>Access was read-only and lasted about 26 minutes on 3 August. The attacker did not modify, disable, or create any checks, alerts, or settings.<br>The attacker made no changes to your account, users, or API keys, and created no new ones.<br>The attacker did not access our production platform; neither the production database nor the Checkly platform, including check execution infrastructure and alerting, were ever breached.<br>We reviewed our cloud audit logs and found no evidence of anyone reaching the data warehouse from outside our known analytics tooling.<br>What you need to do now<br>Rotate any credentials and secrets added directly to a check configuration (custom headers, cookies, query string parameters, or authorization headers).<br>Rotate all OTEL API keys in use.<br>Review the systems those credentials protect for unauthorized use since 3 August 2026.<br>Reauthorize Vercel, Prometheus, and/or Coralogix integrations<br>If your checks only use Checkly secrets for authentication, you do not need to rotate them. For future security, we recommend storing secrets and sensitive data with Checkly Secrets.<br>What we did<br>We rotated the credentials Metabase uses to reach the data warehouse.<br>We reset all Metabase user passwords, removed the compromised administrator account, and rotated all Metabase API keys.<br>We audited the admin accounts on the instance and confirmed the attacker created none.<br>We verified through cloud audit logs that only our known analytics tooling used the warehouse credentials.<br>We are contacting affected customers directly.<br>What we are changing<br>Rotating credentials fixes the immediate problem. It does not fix the reason this hurt: our analytics environment held more sensitive data and had broader access than it needed. So we are making these changes:<br>Extensive audit of analytics and processing tools to limit the exposure surface of any potentially sensitive data.<br>Improve our sanitization process when storing check configurations and data for analytic processing by scoping down data access so only required information is accessed.<br>Vendor security notices will page our on-call engineers directly.<br>Future product features around proactively communicating to customers when potentially sensitive data is exposed in their check configurations.<br>We will update this post if Metabase's ongoing forensic investigation or our own review turns up anything that changes the guidance above.<br>If you have...