New notices not being created from a triggered automation

Uptime Impact: 1 hour, 2 minutes, and 11 seconds


Last Wednesday, January 10th, at around 4:30 PM, we released some changes which, although unrelated to monitoring automations, inadvertently broke them.

Once we became aware of this, early on January 11th, we immediately rolled back our changes to get things working again while we tried to chase down the details of what we had broken.

We have now found the code which broke things and have corrected it; we have also improved our automated testing process around monitoring integrations to help prevent this kind of thing from happening again.

Sorry to everyone who was impacted, and as always, we've tried our best to learn from our mistakes.

Avatar for Robert Rawlins
Robert Rawlins

We've identified the issue and have implemented a fix. Monitoring automations are now triggering and publishing correctly. Thanks for your patience during this incident.

Avatar for Nic Coates
Nic Coates

We are investigating an issue with monitoring automation, specifically inbound mail, where the automation is triggered, but notices are not created or published.

We will provide updates as we continue to investigate the issue.

Avatar for Nic Coates
Nic Coates
Began at:

Affected components
  • Monitoring Automation
    • Inbound Mail