Healthchecks.io Status - Incident history

Welcome to the Healthchecks.io status page. If there are interruptions to service, we will post a report here, and on our Mastodon account.

Past incidents

May, 2024

Apr, 2024

Mar, 2024

May, 2024
1. Database connectivity isssueNotification SenderPing APIDashboard
  Started: 3 May, 13:24 UTC
  Duration: 1 hour, 4 minutes
  We're investigating an issue with our database.
  Investigating: 3 May, 13:39 UTC
  We have failed over to a standby database instance.
  Verifying: 3 May, 13:46 UTC
  The systems are back up and working through a backlog of outgoing notifications (caused by pings missed during the downtime).
  Resolved: 3 May, 14:29 UTC
  All systems are operational again. We are currently setting up a new standby database server, to replace the standby that was promoted to primary.
  
  The root cause of the outage was hardware failure: a broken power supply of one of the neighboring servers triggered a fuse in the database server's rack.
Apr, 2024
1. Network issuesPing APIDashboard
  Started: 29 Apr, 11:09 UTC
  Duration: 48 minutes
  We are currently experiencing intermittent network connectivity issues, caused by a networking infrastructure fault in our hosting provider's network.
  
  Status updates about the underlying issue: https://status.hetzner.com/incident/d3972134-e340-4082-b0ef-6ede4009c5a9
  Resolved: 29 Apr, 11:57 UTC
  The issue is resolved.
2. Network fault affecting one of our load balancer'sPing APIDashboard
  Started: 26 Apr, 12:55 UTC
  Duration: 31 minutes
  There's a network fault in our hosting provider's network, affecting one of our load balancers. The load balancer's DNS records were automatically removed.
  
  The network fault: https://status.hetzner.com/incident/35ca1da9-49fb-4899-8ba6-86425f57431e
  Resolved: 26 Apr, 13:26 UTC
  The issue is resolved.
3. Increased ping latency from clients in Hetzner networkPing APIDashboard
  Started: 17 Apr, 19:45 UTC
  Duration: 9 hours, 20 minutes
  We are currently seeing ping request latencies in the multi-second range from clients hosted in Hetzner network.
  
  Hetzner is currently performing backbone maintenance, which is the likely cause of the latency.
  Resolved: 18 Apr, 05:06 UTC
  The issue is resolved.
Mar, 2024
1. Webhook delivery failuresNotification Sender
  Started: 11 Mar, 07:45 UTC
  Duration: 11 minutes
  Healthchecks.io was not delivering webhook notifications for about 10 minutes starting from 07:45 UTC. The issue was caused by a bad deploy, and has now been fixed.
  Resolved: 11 Mar, 07:56 UTC
  The issue is resolved.