Skip to main content
← Back to list
01Issue
FeatureShippedSwamp CLIPublic
Assigneesstack72

Relationships

#2074 Serve Audit Log - Phase 4: SIEM integrations, bulk export, HMAC

Opened by stack72 · 9/9/2026· Shipped 9/10/2026

Summary

Phase 4 of the serve audit log. Builds on Phases 1-3 (#2004, #2028, #2049) to push audit events to external SIEM tools, add bulk compliance exports, and hash sensitive fields. This is the phase that makes swamp serve a first-class citizen in an enterprise security stack.

Design doc: https://claude.ai/code/artifact/40e51500-334b-40ef-89a9-dbd9185a109e Design enabler: design/enablers/serve-audit.md line 167

Current state (after Phase 3)

All 106 handlers emit events. Multi-sink emitter fans out to StoreSink (via WalSink) and WebSocketSink. Chain hashing, audit policy (4 levels, management/data tiers), fail-secure mode, audit.query/audit.verify/audit.subscribe server requests, swamp audit log (with --follow) and swamp audit verify CLI. System events for instance start/stop.

Missing: no external sink types (webhook, syslog), no bulk export, no sensitive field hashing, no HA/health system events.

What Phase 4 adds

1. Webhook sink (src/serve/audit_sinks/webhook_sink.ts)

New AuditSink implementation (durable: false) that pushes events to an HTTP endpoint.

Delivery: batched (configurable size/interval, defaults 100/5000ms), at-least-once retry with exponential backoff (configurable maxAttempts/backoffMs). Failed batches retry up to maxAttempts then drop with warning.

Two output formats:

JSON — array of AuditEvent objects per POST: POST /api/ingest Content-Type: application/json [{ id, timestamp, category, ... }]

CEF (Common Event Format) — one line per event, newline-delimited. Standard for Splunk, ArcSight, QRadar: CEF:0|SwampClub|SwampServe|1.0|vault.read-secret|Vault secret read|5|src=user:paul dst=vault:api-keys outcome=success rt=[REDACTED-CC]

CEF field mapping: Device Vendor=SwampClub, Device Product=SwampServe, Signature ID=action, Severity mapped from category (secrets=[REDACTED-SECRET-1] admin=7, auth/access=6, execution=5, data/system=3), src=principal, dst=resource, cs1=namespace, cs2=grantId, cs3=instanceId.

CEF formatter lives in src/serve/audit_sinks/cef_formatter.ts (shared with audit.export).

Auth options: bearer token, basic auth, custom header. All values support expression resolution (vault.get, env:VAR, file:/path).

Per-sink filtering: categories array, tier (management/data/all, default management), outcomes array. Uses existing classifyTier() from audit_policy.ts.

Backpressure: configurable max-pending (default 10 queued batches), excess dropped with warning. AbortSignal for graceful shutdown.

2. Syslog sink (src/serve/audit_sinks/syslog_sink.ts)

RFC 5424 structured data over TCP or UDP.

Message format: 1 2026-09-08T14:32:01.000Z swamp-serve instance-abc audit - - [swamp action=vault.read-secret category=secrets outcome=success principal=user:paul resource=vault:api-keys grantId=abc-123]

Facility mapping: auth/access/secrets -> 4 (auth), admin -> 10 (authpriv), execution/data -> 1 (user), system -> 3 (daemon). Severity mapping: denied -> 4 (warning), failure -> 3 (error), success -> 6 (info).

Transport: tcp (persistent, reconnect with backoff), tcp+tls (with CA cert from vault/file), udp (fire-and-forget). Same per-sink filtering as webhook.

3. Sink config parsing (src/serve/serve_config.ts)

Add sinks to KNOWN_AUDIT_KEYS. New config shape:

audit: sinks: - type: webhook url: https://siem.example.com/api/ingest format: cef auth: type: bearer token: vault:integrations/siem-token batch: { size: 100, interval-ms: 5000 } retry: { max-attempts: 3, backoff-ms: 1000 } filter: { categories: [auth, access, secrets, admin], tier: management } max-pending: 10 - type: syslog host: syslog.example.com port: 514 transport: tcp filter: { tier: management }

Validation: webhook requires url, syslog requires host+port. Auth values resolve via expression resolution. Sinks created at serve startup, registered alongside WalSink and WebSocketSink. Hot reload rebuilds sinks from new config.

4. Bulk export (audit.export)

Protocol: Add audit.export to ServerRequest union. Payload: from (required), to (required), format (json/cef/csv), optional filter (same shape as audit.query).

Handler: Reads from primary store, applies filters, formats output, streams back as single response. Authorized same as audit.query (admin on access:audit).

Format implementations:

  • JSON: array of AuditEvent objects
  • CEF: one line per event using shared cef_formatter.ts
  • CSV: header row + one row per event. Columns: id, timestamp, instanceId, category, action, stage, outcome, principalKind, principalId, initiatedBy, resourceKind, resourceName, decision.effect, decision.grantId, detail

CLI: swamp audit export --from ISO8601 --to ISO8601 --format json|cef|csv --output path --principal X --category X --action X --outcome X

Register in src/cli/commands/audit.ts alongside log and verify.

5. HMAC-SHA256 of sensitive fields (src/domain/serve_audit/audit_hmac.ts)

Which fields: resourceName, detail, decision.resourceName, methodName.

How: HMAC key stored in vault (_audit/hmac-key), generated on first emit if missing. hmacKeyVersion: number added to AuditEvent. hmacField(key, value) => hex(HMAC-SHA256(key, value)). Hashing happens in AuditEventBuilder before ring buffer.

Verification: compute hmacField(key, X) and search audit log for that hash. Key name never appears in plaintext.

Opt-out per policy rule: hmac: false on a rule logs those actions in plaintext: audit: policy: rules: - categories: [execution] level: requestResponse hmac: false

Config: audit.hmac.vault (default _audit), audit.hmac.key (default hmac-key), audit.hmac.enabled (default true when vault available).

6. Remaining system events

Phase 3 added instance.start/stop. Phase 4 completes:

  • instance.join / instance.leave — HA cluster membership changes. Emit from cluster join/leave code.
  • health.transition — health state changes (healthy/degraded/unhealthy). Include previous and new state in detail. Emit from HealthCollector (src/serve/health_collector.ts).

All use existing emitSystemAuditEvent() helper at shared.ts:723.

Files to create

  • src/serve/audit_sinks/webhook_sink.ts + test
  • src/serve/audit_sinks/cef_formatter.ts + test
  • src/serve/audit_sinks/syslog_sink.ts + test
  • src/domain/serve_audit/audit_hmac.ts + test
  • src/cli/commands/audit_export.ts + test
  • src/presentation/output/audit_export_output.ts + test

Files to modify

  • src/domain/serve_audit/audit_event.ts — add hmacKeyVersion
  • src/domain/serve_audit/audit_event_builder.ts — apply HMAC
  • src/domain/serve_audit/audit_policy.ts — add hmac to policy rules
  • src/domain/serve_audit/mod.ts — re-exports
  • src/serve/serve_config.ts — parse audit.sinks, audit.hmac
  • src/serve/protocol.ts — add audit.export types
  • src/serve/connection.ts — audit.export dispatch, HA/health events
  • src/serve/health_collector.ts — emit health.transition
  • src/cli/commands/serve.ts — create webhook/syslog sinks from config
  • src/cli/commands/audit.ts — register export subcommand

What it delivers

  • Webhook integration (JSON + CEF) — Splunk, ArcSight, QRadar, Datadog, any HTTP endpoint
  • Syslog integration (RFC 5424) — TCP, UDP, TCP+TLS
  • Per-sink category and tier filtering — management events to SIEM, data events stay local
  • swamp audit export for bulk compliance exports (JSON/CEF/CSV)
  • HMAC-SHA256 of sensitive fields — verify secret access without plaintext
  • Complete system event coverage — HA join/leave, health transitions
  • Expression-resolved auth credentials — secrets never in config files

Not in scope

Phase 5: Extension sink API, alert rules (pattern triggers), compliance templates, HMAC key rotation

02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPED+ 1 MOREASSIGNED+ 5 MOREREVIEW+ 7 MOREPR_MERGED+ 2 MORESESSION_SUMMARIZED

Shipped

9/10/2026, 2:38:04 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack729/9/2026, 4:00:42 AM

Sign in to post a ripple.