Skip to main content
← Back to list
01Issue
FeatureIn ProgressSwamp CLIPublic
Assigneesstack72

Relationships

#2103 Serve Audit Log - Phase 5: Extension sinks, alerting, compliance

Opened by stack72 · 9/10/2026

Summary

Phase 5 of the serve audit log. Builds on Phases 1-4 (#2004, #2028, #2049, #2074) to make the audit system extensible, proactive, and compliance-ready. This is the long-tail phase that deepens the moat — the audit log already works end-to-end; Phase 5 opens it up to the extension ecosystem, adds pattern-based alerting, and provides ready-made compliance outputs.

Design doc: https://claude.ai/code/artifact/40e51500-334b-40ef-89a9-dbd9185a109e Design enabler: design/enablers/serve-audit.md line 202

Current state (after Phase 4)

The audit log is production-ready: all 106 handlers emit events, durable storage with WAL and chain hashing, audit policy with 4 detail levels, fail-secure mode, queryable via CLI (swamp audit log/verify/export), streamable via WebSocket (audit.subscribe, --follow), events push to external SIEMs via webhook (JSON/CEF) and syslog (RFC 5424), sensitive fields HMAC-hashed, system events for instance lifecycle and health transitions.

What Phase 5 adds

1. Extension sink API

Let extension authors register custom audit sinks. The same extension system used for datastores, vaults, and reports gains an audit-sink extension point.

An extension sink implements the AuditSink interface (name, durable, write/flush/close) and is loaded at serve startup from config:

audit: sinks: - type: extension extension: @acme/kafka-audit-sink config: brokers: [kafka-1:9092, kafka-2:9092] topic: swamp-audit-events filter: tier: all

This enables integrations we do not build ourselves: Kafka, Elasticsearch, Azure Event Hubs, PagerDuty, custom internal systems. The extension receives the same typed event stream as built-in sinks.

What needs building:

  • Extension sink loader: resolve extension, instantiate AuditSink from its exported factory
  • Config parsing: type: extension in audit.sinks array, with extension name and arbitrary config
  • Sink lifecycle: create at startup, swap on hot-reload, graceful close on shutdown
  • Extension contract: documented interface that extension authors implement (AuditSink + factory signature)
  • Testing contract: a conformance test suite in packages/testing that extension authors can run against their sink

Follow the same patterns as datastore extensions (ControlPlaneStore adapter) and vault extensions.

2. Alert rules

Configurable triggers on audit event patterns. This turns the audit log from a passive record into an active detection system.

An alert rule defines a pattern (category, action, outcome, principal) and a threshold (count within a time window). When the threshold is breached, the alert fires and delivers a notification.

audit: alerts: - name: brute-force-auth description: Multiple denied auth attempts match: category: auth outcome: denied threshold: count: 5 window-seconds: 60 action: webhook: https://pagerduty.example.com/alert # or: log (emit a system audit event)

What needs building:

  • Alert rule engine: in-memory sliding window counters per rule, evaluated on each event
  • Alert rule config parsing and validation
  • Alert delivery: webhook POST (reuse webhook infrastructure from Phase 4), or emit a system-category audit event for the alert itself
  • Alert state: which rules are currently triggered, cooldown period to prevent alert storms
  • CLI: swamp audit alerts (list active alert rules and their current state)
  • Server request: audit.alerts (query alert rule state)

Design considerations:

  • Rules evaluate in the AuditEmitter drain loop, after chain hashing, before sink fan-out
  • Window counters are per-instance (no cross-instance coordination in Phase 5)
  • Alert events are themselves audit events (category: system, action: alert.fired) so they flow through sinks and are queryable
  • Cooldown: once an alert fires, it does not re-fire for the same rule until the window passes without breaching the threshold

3. Compliance report templates

Pre-built audit report definitions that produce formatted output for common compliance asks. These use the existing report extension system.

Templates:

  • Access review: who has access to what, based on grant events and current policy state. Covers: all principals, their groups, effective permissions, last activity timestamp.
  • Secret access log: every vault.read-secret event in a time range, grouped by principal and key. Answers: who accessed which secrets and when.
  • Change history: all configuration-category events (model/workflow/vault create/edit/delete) in a time range. Answers: what changed, who changed it, when.
  • Denied access report: all denied events grouped by principal, rule, and resource. Answers: who is being blocked and why.
  • System event log: all system-category events (instance lifecycle, health, HA, alerts). Answers: what happened to the infrastructure.

What needs building:

  • Report model definitions using the existing report extension system
  • Each template queries via audit.query with appropriate filters
  • Output in both log mode (formatted table) and json mode (structured data)
  • Time range parameterization (--from, --to, or preset: last-7d, last-30d, last-quarter)

4. HMAC key rotation

Phase 4 added HMAC hashing with a single key version. Phase 5 adds rotation:

  • swamp audit rotate-key: generates a new HMAC key version, stores it in the vault alongside the old one
  • New events use the new key version (hmacKeyVersion increments)
  • Old key versions are retained indefinitely for verification of historical events
  • audit.verify checks HMAC consistency using the key version recorded on each event

What needs building:

  • CLI command: swamp audit rotate-key
  • Server request: audit.rotate-key (generates new key version, returns version number)
  • Key version registry: vault stores all versions keyed by version number
  • AuditEventBuilder: look up current key version at emit time
  • AuditQueryService: verify HMAC using the event's recorded key version

Follows the AWS KMS model: old key versions live indefinitely, the version on each event tells you which key to verify with. No re-hashing of historical events.

5. Streaming bulk export

Phase 4's audit.export loads all matching events into memory and returns them as a single response. For large time ranges this can exceed memory limits. Add streaming export:

  • Server streams events as they are read from the store, one batch at a time
  • CLI writes incrementally to --output file or stdout
  • Backpressure: server pauses reading when the client is slow to consume

This is an enhancement to the existing audit.export, not a new request type.

6. Hot-reload of external sinks

Phase 4 sinks are created at startup. When serve.reload is triggered, external sinks (webhook, syslog, extension) should be rebuilt from the new config:

  • New sinks created from new config
  • Old sinks flushed and closed
  • Emitter swaps sink list atomically
  • WebSocket sink persists across reload (subscriptions are per-connection, not config-driven)

Files to create

  • Extension sink loader + contract definition
  • packages/testing audit sink conformance suite
  • Alert rule engine + config (src/domain/serve_audit/audit_alerts.ts)
  • Alert delivery (webhook + system event)
  • Compliance report templates (5 report definitions)
  • src/cli/commands/audit_rotate_key.ts
  • src/cli/commands/audit_alerts.ts

Files to modify

  • src/serve/serve_config.ts — parse extension sinks, alert rules
  • src/serve/protocol.ts — add audit.rotate-key, audit.alerts types
  • src/serve/connection.ts — dispatch new request types, hot-reload sink swap
  • src/domain/serve_audit/audit_emitter.ts — alert rule evaluation in drain loop, sink hot-swap
  • src/domain/serve_audit/audit_hmac.ts — multi-version key lookup
  • src/domain/serve_audit/audit_event_builder.ts — current key version lookup
  • src/domain/serve_audit/audit_query_service.ts — HMAC verification with versioned keys
  • src/cli/commands/audit.ts — register rotate-key and alerts subcommands
  • src/cli/commands/serve.ts — extension sink loading, hot-reload sink rebuild

What it delivers

  • Extension sink API: Kafka, Elasticsearch, or any custom sink via the extension system
  • Pattern-based alerting: detect brute-force auth, suspicious secret access, unusual admin activity
  • Compliance reports: access reviews, secret access logs, change history — ready for auditors
  • HMAC key rotation: rotate without downtime, verify historical events with old keys
  • Streaming export: handle large time ranges without memory exhaustion
  • Hot-reload sinks: change sink config without restarting serve
02Bog Flow
✓OPEN✓TRIAGED◉IN PROGRESS○SHIPPED+ 1 MOREASSIGNED+ 5 MOREREVIEW+ 3 MOREVERIFICATION_STARTED

In Progress

9/10/2026, 8:38:46 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack729/10/2026, 8:17:04 PM

Sign in to post a ripple.