Zabbix problem to host investigation
A Zabbix problem on a crash-looping service brings signed host evidence in under a second, and Claude finds the config change behind it.
The scenario
Two web servers, web-1 and web-2, ran a small service, orders-api, with systemd set to restart it whenever it stops, and Zabbix tracked the restart count. We broke web-2 the way a bad deploy would: one line in the service's env file moved it onto a port another program already held, so it could not start.
The alert
A Zabbix trigger raised a High problem, orders-api is restarting repeatedly, on web-2. A standard Zabbix webhook media type, with its key kept in a secret macro, sent the problem to the Secronyx alert bridge.
At a glance
Zabbix raises a High problem: orders-api is restarting repeatedly on web-2
A Zabbix webhook sends the problem to the Secronyx alert bridge
The bridge runs a service-failure-triage playbook of 13 read-only checks
web-2 signs every result about half a second later
Claude follows up through Secronyx, comparing web-2 with web-1
Claude reports a misconfiguration and leaves the fix to a person
What Secronyx gathered, and how it was restricted
What it gathered from the host
- Recent configuration file changes, with the time each changed
- Service failures and the error the service logged on each start
- Listening ports on the host
- Logins, deploys and package changes around the time of the problem
- The same checks on the twin host, for comparison
How it was restricted
- Read-only checks: no check changes the host, so the fix was left for a person to make
- Reading the env file or the process's environment needs a scope this policy does not give the AI, so Claude could not read the new port number
- A label policy decides what each caller may see; the card-data host is reserved for the security team
- Each host checks a single-use pass signed by the commander and writes every call to its own tamper-evident audit log
- Claude had one tool, the commander: no shell, no file access and no Zabbix access beyond the data in its first prompt
What the AI concluded
Claude's verdict was misconfiguration. It named the changed file and the time, the healthy listener stopping 17 milliseconds later, the address-in-use error on every start since, and that web-1 still runs the service on its usual port. Its fix was to restore the port setting to match web-1 and confirm the restart counter stops, and it asked a person to confirm the change was sanctioned.
The proof
Figures from the recorded run, as reported in the walkthrough.
< 1 s
from the Zabbix notification to signed host evidence
26 calls
made by Claude, all to Secronyx
28 of 28
signed findings verified offline; changing one letter makes verification fail
17 ms
between the config change and the healthy listener stopping, as Claude reported

See every step
The walkthrough has the setup, the full story with screenshots, the evidence and how to run it yourself.
Product names belong to their owners and are used only to say what each demo connects to. Secronyx is not affiliated with, sponsored or endorsed by them.