Статус

Предыдущие инциденты

1. Май, 2024

   1. Database connectivity isssueNotification SenderPing APIDashboard

      Стартовал: 3 Май, 13:24 UTC

      Продолжительность: 1 час, 4 минуты

      We're investigating an issue with our database.

      Расследование: 3 Май, 13:39 UTC

      We have failed over to a standby database instance.

      Проверка: 3 Май, 13:46 UTC

      The systems are back up and working through a backlog of outgoing notifications (caused by pings missed during the downtime).

      Решено: 3 Май, 14:29 UTC

      All systems are operational again. We are currently setting up a new standby database server, to replace the standby that was promoted to primary.
      
      The root cause of the outage was hardware failure: a broken power supply of one of the neighboring servers triggered a fuse in the database server's rack.

2. Апр, 2024

   1. Network issuesPing APIDashboard

      Стартовал: 29 Апр, 11:09 UTC

      Продолжительность: 48 минут

      We are currently experiencing intermittent network connectivity issues, caused by a networking infrastructure fault in our hosting provider's network. 
      
      Status updates about the underlying issue: https://status.hetzner.com/incident/d3972134-e340-4082-b0ef-6ede4009c5a9

      Решено: 29 Апр, 11:57 UTC

      The issue is resolved.
   2. Network fault affecting one of our load balancer'sPing APIDashboard

      Стартовал: 26 Апр, 12:55 UTC

      Продолжительность: 31 минута

      There's a network fault in our hosting provider's network, affecting one of our load balancers. The load balancer's DNS records were automatically removed.
      
      The network fault: https://status.hetzner.com/incident/35ca1da9-49fb-4899-8ba6-86425f57431e

      Решено: 26 Апр, 13:26 UTC

      The issue is resolved.
   3. Increased ping latency from clients in Hetzner networkPing APIDashboard

      Стартовал: 17 Апр, 19:45 UTC

      Продолжительность: 9 часов, 20 минут

      We are currently seeing ping request latencies in the multi-second range from clients hosted in Hetzner network.
      
      Hetzner is currently performing backbone maintenance, which is the likely cause of the latency.

      Решено: 18 Апр, 05:06 UTC

      The issue is resolved.

3. Мар, 2024

   1. Webhook delivery failuresNotification Sender

      Стартовал: 11 Мар, 07:45 UTC

      Продолжительность: 11 минут

      Healthchecks.io was not delivering webhook notifications for about 10 minutes starting from 07:45 UTC. The issue was caused by a bad deploy, and has now been fixed.

      Решено: 11 Мар, 07:56 UTC

      The issue is resolved.