#2828 Docs: telemetry reference — the envelope we send and our redaction policy

Problem

The manual has no telemetry page. Users and security reviewers can't find out:

• what the CLI sends
• when it sends it
• what is redacted before anything leaves the machine
• how to turn it off, and where each opt-out applies

Today the only way to answer those questions is to read src/cli/telemetry_integration.ts or point telemetryEndpoint at a local HTTP server and inspect the payloads. #1597 covers the missing page from the serve/scheduling angle. This issue asks for the reference entry itself: the envelope schema and the redaction policy, written so an enterprise security review can rely on it.

Proposed entry

A Reference page, content/manual/reference/telemetry.md, with an Explanation section or a separate page for the "why". It should contain the following sections.

1. Transport

• Event type cli_invocation, one per CLI invocation. Workflow runs also emit one child event per model-method step, linked by parentInvocationId.
• POST <endpoint>/ingest with Content-Type: application/json.
  • A single event is posted as a bare object. A batch is posted as { "events": [ ... ] }.
  • The CLI flushes the oldest 25 unsent entries at the end of each command, with a 2s ceiling. swamp serve flushes every 60s.
  • Only 202 counts as delivered.
• Headers: User-Agent. When signed in, it also sends x-api-key, and the server resolves that to the account. The body never names the account, username or email.
• The connecting IP address is not part of the payload. The ingest service necessarily sees it at the transport layer, but does not log or store it. The page should say this explicitly.
• Default endpoint: https://telemetry.swamp-club.com. Override order: SWAMP_TELEMETRY_ENDPOINT env var, then .swamp.yaml telemetryEndpoint, then the localhost auto-detect from the auth server URL, then the default.
• Local spool: one JSON file per invocation in <config>/telemetry/, where <config> is $SWAMP_HOME/config, $XDG_CONFIG_HOME/swamp or ~/.config/swamp. The file is written before any send is attempted.
  • Sent files are deleted, or kept as .flushed.json when telemetryKeepFlushed: true.
  • Unsent files are retried on later runs and deleted after 7 days.

2. Envelope

The schema, with every field documented: type, whether it is optional, where the value comes from, and an example.

{
  "event": "cli_invocation",
  "distinct_id": "string",
  "insert_id": "string",
  "properties": {
    "id": "string",
    "invocation": {
      "command": "string",
      "subcommand": "string",
      "commandPath": ["string"],
      "args": ["string"],
      "optionKeys": ["string"],
      "globalOptions": ["string"]
    },
    "result": {
      "status": "success | error | user_error",
      "errorType": "string",
      "errorMessage": "string",
      "exitCode": 0
    },
    "startedAt": "ISO-8601",
    "completedAt": "ISO-8601",
    "durationMs": 0,
    "swampVersion": "string",
    "denoVersion": "string",
    "platform": "string",
    "invocationContext": {
      "configuredAiTools": ["string"],
      "detectedAiTool": "string",
      "agentSessionDetected": false,
      "isInteractive": false,
      "externalDatastoreConfigured": false,
      "datastoreType": "string",
      "externalVaultConfigured": false
    },
    "parentInvocationId": "string",
    "workflowContext": {
      "workflowName": "string",
      "runId": "string",
      "jobName": "string",
      "stepName": "string",
      "modelType": "string",
      "executor": "string"
    },
    "triggerSource": "schedule | webhook | api",
    "initiatedBy": "string",
    "$repo_id": "string"
  }
}

Points the field table must make explicit:

• Identifiers:
  • distinct_id is a random per-install UUID from <config>/identity.json.
  • insert_id and properties.id are a random per-event UUID, used for server-side dedup.
  • $repo_id is a random per-repo UUID from .swamp.yaml.
  • runId and parentInvocationId are random per-run and per-invocation UUIDs.
  • None of these is derived from machine, user or customer data.
• invocation:
  • commandPath is the resolved command from swamp's own command tree, e.g. ["model","type","describe"]. It is new with #2817.
  • command and subcommand are its first two words, kept for compatibility.
  • optionKeys holds flag names only, never flag values.
  • globalOptions is the recognised global flags (--json, --quiet, …).
• result:
  • errorMessage is the first line of the error, after redaction (see below).
  • errorType is the error class name.
• invocationContext: booleans and tool names derived from an allowlist of agent-detection environment variables. No environment values are sent.
• workflowContext, triggerSource and initiatedBy: present only on workflow step events and swamp serve-triggered runs.
• Not collected, ever:
  • stdout and stderr
  • environment variable values
  • file contents
  • model and resource data
  • vault contents
  • hostnames or IP addresses of the machine
  • the absolute repository path

3. Redaction policy

State the policy as a rule, then as a table:

Telemetry keeps what kind of thing was done and what it was called. It never keeps what is inside. Names are labels, so they are kept. Paths, values, queries over data, and output are content, so they are redacted or never collected.

Also include:

• A worked example: one real command, e.g. swamp model method run my-model execute --input run="..." with a failing --repo-dir, shown next to the exact event it produces.

• A short "how to verify this yourself" box:

  1. Point telemetryEndpoint at a local listener (nc -l 8080, or a 10-line server that returns 202).
  2. Run any command.
  3. Read the body.

  Alternatively, point it at a closed loopback port and read the spooled files in <config>/telemetry/. Nothing is transmitted.

4. Turning it off

Recommend SWAMP_NO_TELEMETRY=1 as the single machine-wide switch. State that repo settings apply only to commands that resolve the repo; a bad --repo-dir or a command run outside the repo uses user-level config. Also cover:

• The update check: SWAMP_NO_UPDATE_CHECK, at most once per 24h, a HEAD request to the artifacts host.
• The other outbound calls: the extension auto-resolver (trustedCollectives: [] disables it), and whoami when SWAMP_API_KEY is set.

Acceptance criteria

• The page exists and is linked from the manual index and from the serve and scheduling pages (#1597).
• The envelope schema on the page matches TelemetryEntryData and the HTTP sender, including commandPath. Ideally a docs test fails when a field is added to TelemetryEntryData without a matching entry on the page.
• The redaction table matches the behavior shipped in #2817 and is phrased as policy, not implementation, so a reviewer can hold the product to it.
• The opt-out table and its scope caveats are verified against the current CLI.

Related

• #1597: the manual never mentions telemetry. This issue is the reference entry it asks for.
• #2817: the redaction model this page documents (names kept; paths, values and predicates redacted; commandPath).
• #1824: the earlier error-message redaction.
• #1129 and #1233: user-global spool and telemetry.yaml opt-out docs.