Relationships
#1255 Reconcile collective-token CLI activity to members (attribute → bonus → stamp → even-split)
Opened by keeb · 7/18/2026· Shipped 8/24/2026
Problem
swamp-club has exactly one identity-reconciliation path: an anonymous ghost device → a person, via bindIfUnclaimed on whoami, gated to personal API-key auth (routes/api/whoami.ts:56-61).
Collective API tokens (swamp_org_*, collective_apikey) authenticate device activity as an org, but take a separate middleware path that sets session: null and no username (routes/_middleware.ts:344-383). As a result:
- The
whoamibinding never fires for them — the GET handler short-circuits into the collective branch beforehandleGetWhoami(whoami.ts:84-100). identity_maphas no collective dimension — it is{username, distinct_ids[]}, person-keyed only.- At telemetry ingest,
server.ts:236-254resolves usernames only for personal keys, so a collective-tokencli_invocationresolves to no username and is scored as an anonymous ghost — value that accrues to nobody.
Net: all non-publish collective-token device activity (cli_invocation, presence, days_active) is orphaned. Only extension_published currently distributes to members. This is a meaningful blind spot — e.g. HiveMQ's ASDLC factory (~320+ pod device-ids operating under @hivemq) generates large CLI volume that reconciles to nothing, and it is invisible to any identity_map-based "claimed vs anonymous" analysis.
The mechanism (decided)
CLI events attributed to a collective receive a bonus, get stamped, and are evenly distributed to the collective's current members.
This is already how extension publishing works — the pattern is proven and just needs generalizing from extension_published to cli_invocation:
routes/api/v1/extensions/confirm.ts:146-197emits withdistinct_id="collective:<id>",username="@<slug>", and a frozenmember_user_ids/member_usernamessnapshot.services/telemetry/lib/publish-distribution.sql:33-101fans a 1.5× pool evenly across the snapshot into per-membercollective_distributiongrants; thecollective:<id>pseudo-identity keeps only the MV floor (500) as inert provenance.- This also settles the "current members" semantics: frozen-at-stamp (the snapshot is frozen at emit time), not read-time.
Plan — four steps, mapped to existing machinery
- Attribute — server-side, at ingest (no CLI change). The
/ingestendpoint already extracts and verifies the request's API credential and applies it to the whole batch (server.ts:236-254); it just calls the personal resolver. Extend it: on a personal-key miss, tryverifyCollectiveApiKey. For aswamp_org_*token, stamp the batch's events with a collective marker (@sluglabel +collective_id) symmetric to how personal keys stampusername. Preserve the real devicedistinct_id(keeps provenance; lets the device still bind to a person later). This is the linchpin and is ~20 lines symmetric to code already present. - "Current members" dimension. New
swamp.collective_membersClickHouse dimension synced from the Mongomembercollection — a direct analog ofservices/telemetry/lib/identity-map-sync.ts. - Bonus + stamp + even-split. New
cli-collective-distribution.sqlmodeled onpublish-distribution.sql: take collective-attributed successfulcli_invocationactivity, apply the bonus, fan evenly to the current members as immutablecollective_distributiongrants (reuse the existing grant type +@slugrendering), stamped once / idempotent, member snapshot frozen at projection-stamp time. - Stop ghosting. Exclude collective-attributed distinct_ids from the ghost gate / read filter (
lib/infrastructure/clickhouse/clickhouse-score-reads.ts:72,184-188) so they route to the collective/members and don't also surface as anonymous ghosts. Mirrorspublish-topup.sql:54-55excludingcollective:<id>.
Key engineering delta from the publish template
Frequency. Publishes are ~1.3K/week; CLI is ~1.15M/week. A fixed per-event pool per invocation would explode the grant table and unbound the bonus. The projector must roll up to (collective, day) first, then split, and the bonus must be a rate, not a fixed per-event pool.
Open decisions
- Bonus shape/size — the only genuinely undecided piece. Publish uses fixed pools (75k first / 7.5k update) × 1.5. For CLI, propose either a multiplier on the collective's minted daily activity grant (e.g. 1.25×) or a fixed per-active-day pool per collective (then split), so the bonus can't scale with raw invocation volume.
- distinct_id handling — preserve device id + stamp collective label (recommended, keeps provenance) vs re-key to
collective:<id>like publish. - Backfill — go-forward only (past CLI events carry no token marker, so they are unattributable without the deanon workflowContext-namespace hack), or a coarse one-time backfill.
- Abuse bound — even-split to current members is sybil-able (add sock-puppet members to farm future bonuses); frozen-at-stamp caps retroactive gaming. Cap eligible members, or accept and monitor?
Alignment note
Bonusing collective CLI usage incentivizes teams to route work through collective tokens — exactly the paying (Team/Business) shape. Reputation ≠ seats, so this stays consistent with "agents/CI never count" for pricing.
Shipped
No activity in this phase yet.
dougschaefer commented 7/28/2026, 1:16:26 AM
Second concrete case for this, with a wrinkle that I think strengthens the argument: the current behavior actively penalizes doing identity correctly.
We're standing up a production swamp serve host — scheduled workflows on cron plus agent and runbook traffic arriving over --server. It authenticates with a collective token (swamp_org_*) rather than a personal API key, and that was a deliberate architectural decision, not a convenience one:
- A long-lived orchestrator authenticating as a named human means the server can act as that person indefinitely, and offboarding that person breaks production.
- There's no delegation/OBO (see #1069), so a personal key on the box would attribute every agent's and every user's action to one human in a way that's actively misleading in an audit.
- The box is intended to be pointed at customer environments eventually, which makes a personal credential on it a non-starter.
So we did the right thing — and the consequence is that every scheduled run and every agent-initiated workflow on that host generates cli_invocation volume that resolves to no username and scores as an anonymous ghost. It credits neither the operator nor the collective.
The perverse incentive is the part worth fixing: the only way to make a production automation host "count" today is to put a personal API key on a shared server, which is precisely the thing you'd want an identity system to discourage. HiveMQ's ~320 pod device-IDs are the volume argument; this is the governance argument, and I think it's the more durable one — as more orgs run swamp serve as shared infrastructure rather than a personal tool, the population of orphaned activity grows specifically among the deployments that took identity seriously.
The decided mechanism (bonus → stamp → even-split to current members, generalizing the extension_published path) is exactly right for this shape. Even-split to current members is also the correct semantic for a shared orchestrator, since the automation genuinely is org work rather than any individual's.
One adjacent note in case it's useful when implementing: our host resolves its collective token at service start from a secret store via the VM's managed identity, so the token is never on disk and rotates independently of the machine. If stamping ends up keyed on anything token-derived rather than collective-derived, that rotation shouldn't orphan the history.
Happy to be a test deployment for this once it lands.
Sign in to post a ripple.