Troubleshooting

Every message on this page is quoted from the source of Secronyx: cmd/secronyx/main.go and the service_*.go files beside it, internal/mcp/http.go, server.go, oidc.go, ratelimit.go and the auth_*.go files, internal/sshauth, internal/agent, internal/audit, internal/pathpolicy and internal/netconfig/probepolicy.go. Values that vary (addresses, paths, counts) are shown in angle brackets. The server logs to stderr; on the stdio transport stdout is the protocol channel, so always look at stderr first.

Start-up refusals

These are printed as Error: <message> and the process exits 1.

refusing to listen on "<addr>" without authentication: bind to 127.0.0.1, or configure --token/--oidc-issuer/--auth-server, or pass --allow-unauthenticated for an isolated development network The listener is bound to a non-loopback address with no authentication method. Either bind to 127.0.0.1, or add one of the credential flags (--token, --api-keys-file, --ssh-authorized-keys, --ssh-ca-keys, --tls-client-ca, --oidc-issuer, --auth-server). --allow-unauthenticated (or SECRONYX_ALLOW_UNAUTHENTICATED=1) overrides the check and logs SECURITY WARNING: serving "<addr>" WITHOUT AUTHENTICATION because --allow-unauthenticated is set; anyone who can reach this port can read system state.

refusing to listen on "<addr>" with authentication but without TLS: credentials would cross the network in plaintext; add --tls-cert/--tls-key, bind to 127.0.0.1 behind a TLS-terminating proxy, or pass --allow-unauthenticated for an isolated development network Authentication is configured but --tls-cert and --tls-key are not, and the bind is off-loopback. Inside a container this is the common case, because the process must bind 0.0.0.0 to be reachable through a published port. The compose file publishes on 127.0.0.1 and passes --allow-unauthenticated for exactly that reason; the Helm chart requires mcp.tls or an explicit mcp.allowInsecure: true. See Docker and Compose and Kubernetes and Helm.

bearer token is <n> characters; a static token must be at least 32 characters of random data --token or SECRONYX_TOKEN is too short. This is refused on every bind including loopback, and --allow-unauthenticated does not relax it. Generate one with openssl rand -base64 32.

a client CA bundle (mutual TLS) requires --tls-cert and --tls-key and Error: --mtls-identity-file requires --tls-client-ca Mutual TLS needs the server's own certificate, and the identity file only means something when a client CA is set.

client CA bundle "<path>" contains no certificates The file exists but has no PEM CERTIFICATE blocks. Check that it is the CA bundle, not a key.

Error: --oidc-audience required with --oidc-issuer, Error: --client-id and --client-secret required with --auth-server, Error: --saas-url required with --api-key Incomplete flag groups. OIDC always checks the audience; introspection authenticates to the authorization server with client credentials.

Error: unknown transport: <value> --transport accepts stdio and http only.

Error loading API keys: API key store "<path>" is readable by other users (mode <mode>); chmod 600 it The store contains key hashes and metadata, and any group- or other-read bit is refused, so chmod 600. Other store errors carry the same API key store "<path>": prefix: unsupported API key store version <n> (want 1), duplicate API key id "<id>", API key "<id>": malformed hash, API key "<id>": unsupported hash format.

Error configuring SSH signature auth: sshauth: authorized keys: line <n>: <reason> The reasons are unsupported option "<name>" (only the options the verifier understands are accepted on a line), duplicate key SHA256:<fp>, DSA keys are not accepted, RSA key is <n> bits; at least 2048 required, expires "<v>": use YYYY-MM-DD or RFC 3339, and from "<item>": not a CIDR or IP.

Error configuring mutual TLS: mtls identity file: <path>: identities[<i>] (<name>) grants "*"; wildcard scopes are not permitted in the identity file Wildcards are only allowed on --mtls-default-scopes. Related messages: identities[<i>] has no name, identities[<i>] (<name>) must have exactly one match key, <path>: unsupported version <n> (want 1), invalid SPIFFE trust domain "<td>", and for the CRL mtls CRL: <path>: no PEM block found.

Scope policy: <n> tools not registered (sensitive=false, scopes="<list>") Not an error. With the defaults <n> is 9, the sensitive-scope tools; if it is 530 the --scopes list matched nothing (a typo registers zero of the 530 tools). Scope names are lower-case: core, logs, hooks, sensitive, hardware, resources, state, software, security, triage, windows, storage, network, analytics, alerts, compliance, consumer, enhanced, report.

When the HTTP listener does start it logs MCP HTTP Server starting on <addr> followed by Server URL:, Auth: (the configured methods, or none), and Client CA: when mutual TLS is on. If Auth: does not show the method you expected, the flags did not take.

Authentication failures

A rejected request receives 401 Unauthorized with a WWW-Authenticate: Bearer realm="secronyx", resource_metadata="<server-url>/.well-known/oauth-protected-resource" header and the body {"error":"unauthorized","error_description":"<message>"}. The same message is written to the audit log as an auth/token_validation event with result denied, carrying the text in metadata.error, and every 401 counts towards the lockout. Messages from a chain authenticator are prefixed with its name: api-key:, ssh-signature: or mtls:.

Ten 401 responses from one address inside fifteen minutes lock that address out for fifteen minutes.

Rate limiting

429 Too Many Requests with Retry-After: <seconds> and the body rate limit exceeded means the per-address bucket (20 requests/s, burst 40) is empty or the address is locked out. 503 Service Unavailable with Retry-After: 1 and the body server busy means 32 requests are already in flight. Both are answered before authentication. Behind a proxy without --trust-proxy-headers, every client shares the proxy's bucket. --rate-limit=false disables all of this and logs WARNING: HTTP rate limiting and brute-force lockout are DISABLED.

405 Method Not Allowed with the body Method not allowed is a GET on / or a POST on /health; the MCP endpoint is POST / only.

Authorization and tool lookup

These are JSON-RPC errors with HTTP status 200.

{"jsonrpc":"2.0","id":1,"error":{"code":-32602,"message":"Tool not found","data":"get_env_vars"}}

The tool is not registered. For a sensitive tool, start with --enable-sensitive (or SECRONYX_ENABLE_SENSITIVE=1); for any other, check --scopes. The client cannot distinguish a withheld tool from a misspelt one. Audited as tools/call with result error and error tool not found.

{"jsonrpc":"2.0","id":1,"error":{"code":-32003,"message":"Forbidden","data":"tool \"get_env_vars\" requires scope \"sensitive\""}}

The tool exists but the caller's grants do not cover it. A grant matches as *, <scope> or mcp:tools:<scope>; mcp:tools alone grants nothing. Fix the key record, the scopes= option, the identity file, or the claims the IdP issues. Audited as denied with error scope denied; counted in secronyx_tool_call_errors_total{error_type="scope_denied"}. See Scopes and authorization.

Errors inside tool results

A collector failure is returned as a tool result with isError: true and a text content of Error: <message>, not as a JSON-RPC error.

Error: result too large: <n> bytes exceeds the <cap> byte cap; narrow the query (limit, lines, filters) The default cap is 4 MiB (--max-result-bytes). Use the tool's limit, lines or filter arguments, or raise the cap for a dedicated listener. Audited as result_too_large.

Path policy (get_app_config and the lock-file readers). These arrive in the result's error field rather than as a failed call — get_app_config also sets readable: false and prefixes the text with path refused by policy: . The messages are path is empty, path is outside the allowed directories, path matches a denied pattern (<glob>), path is not a regular file, file exceeds the maximum readable size (<n> bytes, limit 1048576). The path and its symlink target must both be under an allowed root (/etc, /opt, /usr/local/etc, /srv, /var/lib, /var/www, /app on Unix; C:\ProgramData, C:\inetpub, C:\Program Files, C:\Program Files (x86) on Windows; --allowed-paths replaces the list). Key material (**/*.pem, **/*.key, **/.ssh/**, /etc/shadow* and so on) is refused inside any root; the refusal never reads the file.

Probe policy (get_network_latency): the probe's error field reads target refused by probe policy: <target> (<reason>) where the reason is one of cloud metadata address, cloud metadata hostname, loopback address; enable with --probe-allow-loopback, link-local address, multicast or broadcast address, unspecified address, private address, denied host <entry>, not in --probe-allow-hosts, invalid hostname, empty target, no addresses or unresolvable address. Metadata targets stay refused even in allowlist mode. See Network and path policy.

Empty results on the wrong platform. A tool that has no implementation for the host returns an empty, well-formed result rather than an error: get_journal_logs on Windows returns entries: [] with count: 0 and source: "journald". Check the platform column in the Tool reference before treating an empty list as a finding.

Slow tools over HTTP. get_incident_triage_snapshot and get_security_posture_snapshot warn in their descriptions that they can take one to five minutes on macOS. The HTTP server's WriteTimeout is 30 s, so a call that runs longer than that cannot deliver its response over HTTP; use stdio for those tools on macOS, or the narrower triage tools.

Audit subsystem

WARNING: audit provider "default" could not open "<path>" (<cause>); audit events are being written to stderr instead The directory does not exist or is not writable by the server's user. Auditing has not stopped; it has moved to stderr. Create the directory and grant the service user write access, or point --audit-output at a writable path (/dev/stdout in containers). Confirm with the next log line, Audit logging enabled: provider=<name> output=<path>, where provider=stderr means the fallback is active.

Audit verification FAILED: hash chain broken at event <n> (seq=<m>) prev_hash does not match the previous line's hash. At event 1 with seq greater than 1 this is the expected result for a file that has been rotated: the chain continues from the previous file. Concatenate the rotated files in order (oldest first, decompressed) and the live file, then verify the concatenation. Anywhere else it means a line was removed or inserted.

Audit verification FAILED: hash mismatch at event <n> (seq=<m>) The line's content no longer produces its recorded hash; the line was modified.

Audit verification FAILED: failed to parse event at line <n> Truncated or non-JSON line, typically a crash mid-write with buffered output. --audit-sync-write or --audit-buffer-size 0 prevents this at a latency cost.

Audit verification FAILED: stderr audit output cannot be verified; configure a file output Verification needs a file. Also note that verifying a path that does not exist creates an empty file and reports Audit verification OK: 0 events verified.

WARNING: audit logging is DISABLED and WARNING: output redaction is DISABLED; secrets in system state will be returned verbatim --no-audit or --no-redact was passed. Both are deliberate, logged choices; if you did not intend them, check the service unit's arguments.

Agent modes

Polling (--mode polling): Error: --saas-url is required for polling mode; Error: --api-key, --token, --email, or stored credentials required for polling mode (followed by Use --login to authenticate with email/password); Error: no stored credentials found. Run with --login first to authenticate. when --email is given but nothing is stored in <config-dir>/credentials.enc. A failing poll loop exits with Polling error: <message>, where the message is one of poll failed: <status> - <body>, submit failed: ... or heartbeat rejected: ....

Hybrid (--mode hybrid): No credentials found: --api-key, --device-token, --token, or stored credentials required for hybrid mode and No instance ID: --instance-id is required for hybrid mode (or use --api-key for auto-registration). Auto-registration reports Error: auto-registration failed: invalid API key format: must start with 'sk_' (got: <prefix>...) for a malformed key and authentication failed (API key: <prefix>...): <message> - verify the API key exists and hasn't expired for a rejected one. On Windows these lines also go to the Event Log under the service name when the event source is registered. See Hybrid and SaaS mode.

Work items report failures in the result rather than stopping the agent: query "<name>" exceeded deadline of <d>: query exceeded deadline (the deadline is the item's timeout, 30 s by default, 120 s maximum; the result has timed_out set), query "<name>" cancelled before completion: context canceled, query "<name>" panicked: <value>, and unknown query: <name> for a name the executor does not dispatch. The executor runs at most four queries at once; a run of timeouts shows up as back-pressure, not as unbounded load.

Service installer

secronyx service install refuses without privileges: root privileges required - run with sudo on Linux, administrator privileges required - run with sudo on macOS, administrator privileges required - run as Administrator on Windows. Other messages: service <name> already exists at <unit path> (Linux and macOS) or service <name> already exists (Windows); service <name> not installed; service <name> not found (no unit file at <path>); and for --name, service name cannot be empty, service name too long (max 63 characters), invalid service name: must start with a letter and contain only letters, numbers, hyphens, and underscores. See Windows service and Command line.

Containers and Kubernetes

• The binary reads /proc and /sys at their normal paths. The HOST_PROC and HOST_SYS variables that appear in the compose file are not read by the code, so mounting the host's /proc at /host/proc does not change what process tools see; the privileged compose profile uses pid: host for that.
• The image runs as uid 1000 with a read-only root filesystem in the chart; /var/log/secronyx is not writable there, which is why the chart defaults mcp.audit.output to /dev/stdout. Without that override you will see the stderr-fallback warning above.
• The chart refuses to render with mcp.auth.enabled: false unless mcp.allowInsecure: true, and with authentication but no mcp.tls unless the same flag is set. This mirrors the binary's own rules.
• A scope set to false in mcp.scopes is passed by omission from --scopes; sensitive: true additionally passes --enable-sensitive.

Windows

Collectors run powershell with -NoProfile -Command (a few also pass -NonInteractive), so Windows PowerShell must be on PATH. Caller-supplied IIS site and application pool names, event log channels and providers are validated against an allowlist before use; a name with characters outside it is refused by the tool rather than passed to PowerShell.

Getting more signal

• --debug turns on verbose agent-mode logging.
• GET /metrics (authenticated like /) exposes secronyx_tool_calls_total, secronyx_tool_call_errors_total by error_type (not_found, scope_denied, execution_error, result_too_large), secronyx_auth_requests_total and secronyx_http_requests_total.
• secronyx --query <tool> --json runs a collector directly, without the MCP layer, scope policy or tools/call audit; it is the quickest way to tell a collector problem from a transport or authorization problem. See Command line.

Related pages: Configuration reference, Authentication, Audit logging, JSON-RPC API, FAQ, and the Secronyx homepage.