Audit Log and SIEM Forwarding

Hash-chained, append-only#

Every audit event is a row in an append-only audit_log table. Each row records the actor, action, resource type and id, before/after snapshots, client IP, and user agent — and carries two chain fields:

• row_hash — a SHA-256 digest over the row's content
• prev_hash — the row_hash of the previous row

Because each row commits to its predecessor, modifying or deleting any chained row breaks the chain from that point forward — tampering cannot be hidden by editing a single row. Rows written before the chain feature existed carry null hashes and are treated as pre-chain (skipped, not invalid) by the verifier.

Verifying the chain#

The same verification walk is exposed three ways, so integrity is checkable in any deployment shape:

Run the offline check from the backend directory:

cd backend
uv run python scripts/verify_audit_chain.py

Exit codes are deliberate:

• 0 — chain verified intact.
• 1 — chain broken: a chained row was modified or deleted. The first broken row id and reason are logged. Treat this as a security incident and preserve the table before touching anything.
• 2 — verification could not run (database unreachable, bad configuration). Distinct from 1 so automation never mistakes "inconclusive" for either verdict.

What gets audited#

• Mutations across the platform: creating, updating, and deleting connectors, pipelines, tasks, jobs, settings, providers, and tokens — with before/after JSON snapshots.
• Authorization denials on the ETL surface, as etl_authz_denied events carrying the capability, denial reason, and project/environment context (see RBAC).
• Project membership changes — adds, role changes, removals.
• Notification test sends (test email, test license alert).
• Session security events, including refresh_token_reuse_detected (see Authentication).

Read-only endpoints are audit-exempt by design — the log records changes and security signals, not browsing.

Reading and exporting#

• Settings → Audit Log gives admins a filterable view of the full log.
• GET /api/v1/audit-log/export streams the log as NDJSON or CSV — a download endpoint suited to periodic archival pulls.
• Per-project activity feed: GET /api/v1/etl/projects/{id}/audit gives project members a scoped view of their own project's history, without needing instance-admin access.

SIEM forwarding#

For continuous shipping into a SIEM, configure Settings → Audit Forwarder:

• Targets: webhook (one JSON POST per row) or syslog (RFC 5424 over TCP, optionally TLS, octet-counted framing). The target address is stored encrypted like every other secret (see Secrets & encryption), and outbound delivery goes through the same egress guard as all other outbound traffic (see Transport & egress).
• Loss-free cursor. The forwarder tails the audit table with a keyset cursor (last forwarded timestamp + id) and advances it only after a confirmed send. If the SIEM is down, the cursor stalls and rows simply wait, committed, in the database — a downed SIEM delays delivery, it never loses events. A crash mid-batch replays the un-acknowledged tail (at-least-once delivery).
• Cadence. A background tick runs every 60 seconds, forwarding everything past the cursor.
• Observability. The backlog (rows past the cursor) is exported as a metric, so you can alert on a stalled forwarder. On first enable, the cursor seeds to the current chain head — forwarding starts with new events, not a historical flood.

Retention#

Audit rows are excluded from the log-retention sweep — run logs are pruned on schedule, audit rows are not. ABRQ_AUDIT_LOG_RETENTION_DAYS exists as a configuration field but is reserved: no purge consumes it in this version, so plan storage for an ever-growing audit table (the SIEM export above is the pressure valve). Details in retention.