Pro plan from £5 per endpoint per month · see pricing
Secronyx
AWS DevOps AgentGrafana

AWS DevOps Agent investigates with Secronyx

AWS DevOps Agent looks inside the servers through Secronyx's read-only checks and traces a filling disk to a deleted log file still held open.

The scenario

Three EC2 instances ran Secronyx: web-1 and web-2, the web tier, and a database host labelled as holding card data. On web-1 a stand-in service, orders-api, kept writing to a log file that log rotation had already deleted, so the space was still used but the file no longer had a name. Every directory looked small.

The alert

Grafana fired when web-1's root filesystem fell below 25% free. Each alert goes to AWS DevOps Agent, which has the commander registered as a single MCP server, and to the Secronyx Grafana bridge, which runs a fixed check of its own and keeps signed evidence.

At a glance

  1. Grafana fires: web-1's root filesystem is below 25% free

  2. AWS DevOps Agent opens an investigation in the same second

  3. The agent calls Secronyx 41 times, comparing web-1 with web-2

  4. It traces 4.9 GiB to a deleted log file held open by orders-api

  5. It proposes a fix for a person to apply, and applies nothing

What Secronyx gathered, and how it was restricted

What it gathered from the host

  • Disk and filesystem usage
  • Open files, including deleted files still held open by a running service
  • Services, processes and configuration, such as the log rotation rule
  • The same checks on the healthy twin host, for comparison

How it was restricted

  • Fixed read-only checks: none changes a host or returns file contents, and the agent keeps its read-only role
  • A label policy decides which hosts the agent may see; the card-data host is missing from its host list
  • Before a host runs anything, it checks a single-use pass signed by the commander, naming the caller and the rule that allowed the call
  • Every call is in the answering host's own audit log, with the agent, its run and the rule that allowed it
  • Results are signed by the host and verify offline

What the AI concluded

AWS DevOps Agent reported that orders-api was holding open a deleted 4.9 GiB log file, which explained about 98% of the extra space web-1 used compared with web-2. It found the rotation rule that caused it, noticed web-2 has the same bug on a smaller scale, and flagged a configuration change only on web-1 as the likely trigger but unconfirmed. When asked to check the card-data host it got zero hosts back and reported it as unchecked, not clean.

The proof

Figures from the recorded run, as reported in the walkthrough.

41 calls

to Secronyx in under three minutes, with nobody steering

4 min 9 s

from alert to root cause

28 findings

in the disk evidence case, verified offline

0 hosts

returned when the agent was asked to check the card-data host

web-1's own audit log for the disk investigation: one event in full, then 31 calls, each made on behalf of the agent under the devops-agent-demo rule, with tool names blanked
web-1's own audit log for this investigation: every call names the agent, its run and the rule that allowed it.

See every step

The walkthrough has the setup, the full story with screenshots, the evidence and how to run it yourself. Read the full run log

Product names belong to their owners and are used only to say what each demo connects to. Secronyx is not affiliated with, sponsored or endorsed by them.