Session Ingest
@vcjdeboer/session-ingestv2026.07.27.1
01README
Lift your own Claude Science (operon) sessions out of the local operon-cli.db into open, replayable, sealable swamp records: a typed transcript, a turn->execution->artifact->env provenance graph, an immutable content-addressed byte corpus, the FULL ordered cell script, the CS skills the session used, its replayable host.* calls, a Tier-1 /private/tmp input freeze, a per-source external-data inventory, a presence-only credential manifest, and reproducible Docker + Nix environment locks — then seal an order-stable bundle-manifest (witness-digested). With @vcjdeboer/session-execute run-notebook + a host-replay shim, a captured session RE-RUNS outside the app. Reads a disposable scrubbed clone (secret tables dropped and never decrypted; explicit column allow-list; refuses a mid-run session), read-only, deterministic. Anti-lock-in data portability for your own local sessions.
02Models
@vcjdeboer/session-ingestv2026.07.27.1session_ingest.ts
Global Arguments
| Argument | Type | Description |
|---|---|---|
| csRoot | string | |
| orgId | string |
fn inspect(project: string, csRoot: string, orgId: string)
Read-only inspect of a QUIESCENT Claude Science session by name or proj_id; emits a `manifest`. Refuses a DB a running session is actively writing. Never mutates the source, never reads a credential.
| Argument | Type | Description |
|---|---|---|
| project | string | |
| csRoot | string | |
| orgId | string |
fn capture_messages(project: string, csRoot: string, orgId: string)
Capture the VERBATIM, typed, ordered message transcript of a QUIESCENT session into a SENSITIVE `transcript` resource (+ content-addressed `body` files for large content). Deterministic -> witness-sealable. Reads a static clone; source never mutated; never reads a credential.
| Argument | Type | Description |
|---|---|---|
| project | string | |
| csRoot | string | |
| orgId | string |
fn capture_corpus(project: string, csRoot: string, orgId: string)
Take a COMPLETE, IMMUTABLE, content-addressed byte COPY of a QUIESCENT session into a SENSITIVE `corpus` resource + content-addressed `blob` files: ALL artifact_versions bytes + PROJECT-SCOPED workspace files (touched by this project's executions, or under workspace dirs it owns), with drift detection (recomputed sha vs recorded checksum), provenance-of-loss for swept/missing, and a replayable/witnessed stamp. Streams large files (bounded memory); path-safety guards against copying outside the o
| Argument | Type | Description |
|---|---|---|
| project | string | |
| csRoot | string | |
| orgId | string |
fn capture_cells(project: string, csRoot: string, orgId: string)
Freeze the FULL ordered execution sequence of a QUIESCENT session (EVERY cell's source + language + cellIndex) into a SENSITIVE `cells` resource (+ content-addressed `body` files for large source) — the replay SCRIPT that complements the provenance GRAPH. capture_provenance keeps only artifact-linked graph nodes; this keeps the setup/helper cells (that define namespace globals) a full replay needs. Reads a static clone; source never mutated; never reads a credential.
| Argument | Type | Description |
|---|---|---|
| project | string | |
| csRoot | string | |
| orgId | string |
fn capture_host_calls(project: string, csRoot: string, orgId: string)
Freeze a QUIESCENT session's REPLAYABLE host.* calls into a SENSITIVE `host_calls` resource ({method, args, response, isError} in call order; responses inlined from data_inline or resolved from data_ref tape files). Consumed by the session-execute host-replay shim so host.mcp/query_db/... replay offline. SECRET-SAFE: credentials_request responses scrubbed to presence only. Reads a static clone; source never mutated.
| Argument | Type | Description |
|---|---|---|
| project | string | |
| csRoot | string | |
| orgId | string |
fn capture_review(project: string, csRoot: string, orgId: string)
Freeze a QUIESCENT session's INDEPENDENT-REVIEWER verdicts (CS's out-of-band verification_checks: verdict/severity/claim/evidence/reviewerModel, in time order) into a SENSITIVE `review` resource + verdict tally. Seals WHAT the reviewer flagged and WHY so the capture-report renders reviewer detail from sealed data, not just the manifest count. claim/evidence are analysis prose, never a credential. Reads a static clone; source never mutated.
| Argument | Type | Description |
|---|---|---|
| project | string | |
| csRoot | string | |
| orgId | string |
fn capture_notifications(project: string, csRoot: string, orgId: string)
Freeze a QUIESCENT session's PARENT<->CHILD delegation messages (CS `notifications`: task payloads + results exchanged via host.delegate) into a SENSITIVE `notifications` resource + per-type tally — the coordination layer the per-frame transcript doesn't carry. payload is coordination/analysis prose, never a credential. Reads a static clone; source never mutated.
| Argument | Type | Description |
|---|---|---|
| project | string | |
| csRoot | string | |
| orgId | string |
fn capture_extras(project: string, csRoot: string, orgId: string)
Freeze a QUIESCENT session's lower-frequency metadata (remote-compute jobs `compute_usage`, extracted claims incl. unchecked `session_claims`, durable beliefs `memories`, artifact folder structure `artifact_folders`) into one SENSITIVE `extras` resource so nothing is silently dropped. Each table read under its own guard — absent tables degrade to empty + a warning. Reads a static clone; source never mutated.
| Argument | Type | Description |
|---|---|---|
| project | string | |
| csRoot | string | |
| orgId | string |
fn capture_annotations(project: string, csRoot: string, orgId: string)
Freeze a QUIESCENT session's USER ANNOTATIONS — artifact comments + thread bookmarks/highlights (transcript_annotations: kind/anchor/note/offsets/origin/timestamps) — into a SENSITIVE `annotations` resource. anchor_text/note are user prose, never a credential. Reads a static clone; source never mutated.
| Argument | Type | Description |
|---|---|---|
| project | string | |
| csRoot | string | |
| orgId | string |
fn capture_settings(project: string, csRoot: string, orgId: string)
Freeze HOW a QUIESCENT session was run into a `settings` resource — the LLM model(s)+effort, delegation (reviewer/sub-agents), compute target, session timeline, the user capability toggles (memory/delegation/auto-review) and bundled specialists. Counts + flags only, never a credential. Reads a static clone; source never mutated.
| Argument | Type | Description |
|---|---|---|
| project | string | |
| csRoot | string | |
| orgId | string |
fn render_html()
Render the VISUAL companion to the capture-report: run the same report over the sealed facets, convert its markdown to a styled self-contained HTML page, and embed the captured PNG figures inline (from the corpus). Writes a single .html file (default <proj_id>_capture_report.html). Reads only sealed swamp data; nothing from the live app.
fn capture_skills(project: string, csRoot: string, orgId: string)
Freeze the CS SKILLS a QUIESCENT session used into a SENSITIVE `skills` resource (+ each used skill's kernel.py as a content-addressed `body` blob). Skill-loads aren't recorded, so the used set is inferred: a skill is used if any symbol its kernel.py exports appears in the session's cell source. The injected context (apply_figure_style etc.) a replay prepends. Read-only from disk; never a credential.
| Argument | Type | Description |
|---|---|---|
| project | string | |
| csRoot | string | |
| orgId | string |
fn capture_provenance(project: string, csRoot: string, orgId: string)
Reconstruct the turn->execution->artifact->env provenance GRAPH of a QUIESCENT session into a SENSITIVE `provenance` resource (+ content-addressed `body` files for large source/stdout/stderr/env content). The cell node is collapsed into execution (canonical node = execution_log.id). Verbatim, deterministic, injection-gated; reads a static clone; source never mutated; never reads a credential.
| Argument | Type | Description |
|---|---|---|
| project | string | |
| csRoot | string | |
| orgId | string |
fn capture_inputs(project: string, roots: string, referenceRoots: string, referenceOver?: number, maxFileBytes: number, allowSensitiveRoot: boolean, sensitiveRootOptIn: string, hashReferenced: boolean, csRoot: string, orgId: string)
Freeze a session's Tier-1 /private/tmp INPUTS (raw working data OUTSIDE the org tree, days-from-deletion) into `blob` files + a SENSITIVE `inputs` record. Roots come from an ALLOWLIST or, when `roots` is omitted, are AUTO-DERIVED from the session's own files_read/files_written trace (the zero-config standard-flow path) — derived roots are still validated. Roots must resolve under /private/tmp,/tmp (else allowSensitiveRoot + sensitiveRootOptIn); copy-roots + accession files honor maxFileBytes (ov
| Argument | Type | Description |
|---|---|---|
| project | string | |
| roots | string | |
| referenceRoots | string | |
| referenceOver? | number | |
| maxFileBytes | number | |
| allowSensitiveRoot | boolean | |
| sensitiveRootOptIn | string | |
| hashReferenced | boolean | |
| csRoot | string | |
| orgId | string |
fn capture_external(project: string, csRoot: string, orgId: string)
Inventory a session's EXTERNAL-DATA provenance (doc-2 gap 3): aggregate THIS project's host_call_log into a SENSITIVE `external` record — a per-source manifest (MCP server or internal method) with COUNTS + metadata ONLY, call CONTENT never persisted (args_json[1+]/data_inline/data_ref never selected — the MCP server is extracted in SQL via json_extract($[0]) under a json_valid guard; credentials_request/get_user_email counted by name, never parsed). Surfaces the release-pin gap as a fillable per
| Argument | Type | Description |
|---|---|---|
| project | string | |
| csRoot | string | |
| orgId | string |
fn capture_credentials(project: string, csRoot: string, orgId: string)
Inventory a session's credential PRESENCE (#23): which credential PROVIDERS THIS project REQUESTED at runtime (host_call_log.credentials_request) into a SENSITIVE `credentials` record — per provider {requestCount, first/last}, NEVER any secret (secret tables dropped from the read clone; only args_json[0]=provider read under a json_valid guard; args_json[1+] never selected; get_user_email out of scope). Emits org-namespaced vault.get CEL references + a `swamp vault put` provisioning manifest for
| Argument | Type | Description |
|---|---|---|
| project | string | |
| csRoot | string | |
| orgId | string |
fn lock_env(project: string, csRoot: string, orgId: string)
Transform a session's captured conda env snapshot into PORTABLE reproducibility LOCK files (#27, spec §9): per distinct env, a version-PINNED Docker lock (environment.yml + Dockerfile via micromamba — re-solved, NOT build-locked) + a Nix SCAFFOLD (flake.nix, honestly not-exact). Emits content-addressed environment.yml/dockerfile/flake.nix files + a SENSITIVE `lockenv` record indexing them per env; type-stamp starts `witnessed` (unvalidated) until #28 replay validates. PURE TRANSFORM (no build/ru
| Argument | Type | Description |
|---|---|---|
| project | string | |
| csRoot | string | |
| orgId | string |
fn seal()
Seal a captured session (#29): read each captured resource's swamp content checksum in canonical order, assemble the order-stable `bundle-manifest` (items + reproducibility stamp + origin). The independent witness DIGEST over these items is produced by @vcjdeboer/session-witness seal_manifest, wired by the seal-bundle workflow. Reads in-process via queryData; writes only its own bundle-manifest.
fn status()
Read-only reconciliation of what is LIVE on Claude Science (operon-cli.db sessions) vs captured LOCALLY in swamp (per session: sealed | inspected | partial + facets). Buckets into liveAndCaptured / liveNotCaptured / capturedNotLive and flags `dbQuiescent`. NEVER refuses on a pinging DB — a listing needs no consistent snapshot. Writes a repo-wide `status` resource.
Resources
manifest(infinite)— Read-only summary of a Claude Science project's provenance state (counts, frames, reviewer passes, missing artifact files, remote-compute flag) plus a per-session breakdown (each top-level session's headline + own frames + own reviewer checks). Instance key = proj_id. Versioned; sha256 pinning belongs to capture.
status(infinite)— Read-only reconciliation of LIVE Claude Science sessions (operon-cli.db) vs LOCAL swamp captures: per session a state (sealed | inspected | partial) + facets, plus liveAndCaptured / liveNotCaptured / capturedNotLive buckets and a `dbQuiescent` flag. A repo-wide overview, not per-session. Instance key = "status".
cells(infinite)— SENSITIVE: the FULL ordered execution sequence of a session — EVERY cell's source + language + cellIndex (large source offloaded to `body` files). The replay SCRIPT (complements the provenance GRAPH). Instance key = proj_id.
host_calls(infinite)— SENSITIVE: a session's REPLAYABLE host.* calls — each {method, args, response, isError} in call order, responses inlined from data_inline or resolved from data_ref tapes. Consumed by the session-execute host-replay shim. credentials_request responses scrubbed to presence only (no token). Instance key = proj_id.
review(infinite)— SENSITIVE: the INDEPENDENT REVIEWER's verdicts — CS's out-of-band verification_checks ({verdict, severity, claim, evidence, reviewerModel} in time order) + the verdict tally. Seals WHAT was reviewed and WHY, so the capture-report shows reviewer detail from sealed data (not just the manifest tally). claim/evidence are analysis prose, never a credential. Instance key = proj_id.
notifications(infinite)— SENSITIVE: the PARENT<->CHILD delegation messages (CS `notifications`) exchanged within a session's frame tree when it used host.delegate — the coordination layer (task payloads, results) the per-frame transcript doesn't carry, in time order + a per-type tally. payload is coordination/analysis prose, never a credential. Instance key = proj_id.
extras(infinite)— SENSITIVE: the lower-frequency CS metadata tables sealed so nothing is silently dropped — remote-compute jobs (`compute_usage`), the falsifiable claims extracted incl. UNCHECKED ones (`session_claims`), durable agent beliefs (`memories`), and artifact folder structure (`artifact_folders`). Each read under its own guard: a table absent on this build degrades to an empty list + a warning. Instance key = proj_id.
annotations(infinite)— SENSITIVE: the user's own marks on a session — artifact COMMENTS + thread BOOKMARKS/highlights (kind, anchor, note, offsets, origin, timestamps). anchor_text/note are user prose. Instance key = proj_id.
settings(infinite)— HOW the session was run: LLM model(s) + effort, delegation (sub-agents like the reviewer), compute target, the session timeline (first→last frame), the user capability toggles (memory/delegation/auto-review) and bundled specialist agents. Counts + flags only, never a credential. Instance key = proj_id.
visual(30d)— Provenance record for a render_html run: the .html path written, its size, and how many figures were embedded. The HTML itself is a file on disk (the visual companion to the capture-report). Instance key = proj_id.
skills(infinite)— SENSITIVE: the CS SKILLS a session used — each used skill's kernel.py (content-addressed body blob) + the exported symbols that judged it used. The injected context (e.g. figure-style -> apply_figure_style) a replay prepends. Instance key = proj_id.
bundle-manifest(infinite)— The logical, order-stable index of a sealed session: every captured resource (name + swamp content checksum + content-ref) in canonical order, plus the reproducibility stamp (witnessed / replayable-nix / replayable-docker) and origin. The witness DIGEST over these items is produced by @vcjdeboer/session-witness seal_manifest (the seal-bundle workflow). Instance key = proj_id.
transcript(infinite)— SENSITIVE/PRIVATE: the VERBATIM, typed, ordered message transcript of a session (turns + typed content blocks; large bodies / inline images offloaded to `body` files). Instance key = proj_id. Verbatim -> deterministic -> witness-sealable. Contains conversation content; treat as private.
provenance(infinite)— SENSITIVE/PRIVATE: the reconstructed turn->execution->artifact->env provenance GRAPH of a session (artifact/execution/env nodes + typed edges; the cell node is collapsed into execution). Node ids: artifact=artifact_versions.id, execution=execution_log.id, env=env_snapshot_hash. Large source/stdout/stderr/env content offloaded to `body` files. Instance key = proj_id. Verbatim -> deterministic -> witness-sealable.
corpus(infinite)— SENSITIVE/PRIVATE: the immutable, content-addressed byte CORPUS of a session — an index of every artifact_versions file + project-scoped workspace file copied into swamp's own `blob` store (sha, size, drift/unverifiable, present/skipped), plus provenance-of-loss for swept/missing files and a replayable/witnessed type-stamp. Instance key = proj_id. The record that survives a CS sweep/uninstall/upgrade; verbatim -> deterministic -> witness-sealable.
inputs(infinite)— SENSITIVE/PRIVATE: the frozen Tier-1 /private/tmp INPUTS of a session (raw working data outside the org tree, days-from-deletion) — copied small files (sha,size) + by-reference records (path,size) for large/reference-root sets, plus harvested re-fetch accessions (SRA runs, RefSeq GC[AF]_). Captured from a user-supplied ALLOWLIST of external roots (default under /private/tmp). Instance key = proj_id; self-describing (capturedRoots). No CS checksum exists for /tmp, so entries are unverifiable by c
external(infinite)— SENSITIVE/PRIVATE: the gap-3 EXTERNAL-DATA INVENTORY of a session — this project's host_call_log aggregated into a per-external-source manifest (MCP server or internal method) with COUNTS + metadata ONLY (callCount, totalBytes, first/last access, errorCount, a global byMethod histogram), never any call CONTENT (args_json[1+]/data_inline/data_ref never selected; credentials_request/get_user_email counted by name, never parsed). Each mcp source carries a fillable releasePin slot (null at capture)
credentials(infinite)— SENSITIVE/PRIVATE: a PRESENCE-ONLY credential inventory — which credential PROVIDERS a session REQUESTED at runtime (host_call_log.credentials_request), per provider {requestCount, first/last}, NEVER any secret value (the secret tables are dropped from the read clone; only args_json[0]=provider is read under a json_valid guard; args_json[1+] never selected). Emits org-namespaced vault.get CEL references + a `swamp vault put` provisioning manifest for REAL providers only, so a replay re-provision
lockenv(infinite)— SENSITIVE/PRIVATE: portable ENV LOCK artifacts (#27, spec §9) transformed from a session's captured conda env snapshot — per distinct env: a version-PINNED Docker lock (environment.yml + Dockerfile via micromamba; re-solved, NOT build-locked) + a Nix SCAFFOLD (flake.nix, honestly not-exact). Lock files are emitted as content-addressed `environment.yml`/`dockerfile`/`flake.nix` file specs; this record indexes them per env with a type-stamp (starts `witnessed`; #28 replay validates → `replayable-d
Files
body(text/plain)— SENSITIVE/PRIVATE: a content-addressed (sha256) offloaded large text body or inline image bytes referenced by transcript/provenance records via bodyFileRef/imageFileRef/sourceFileRef/packagesFileRef. Keyed by sha (one version per instance) so a version-count GC never evicts a referenced body — a sealable record never dangles.
blob(application/octet-stream)— SENSITIVE/PRIVATE: a content-addressed (sha256) BINARY artifact/workspace file copied verbatim from a CS session by capture_corpus. Keyed by sha (one version per instance) so a version-count GC never evicts a referenced blob. Streamed for large files.
yml(text/plain)— SENSITIVE/PRIVATE: a content-addressed (sha256) reconstructed conda environment.yml emitted by lock_env — exact name=version per conda package + a pip: block + source channels (conda_history.channels). The version-pinned Docker lock's package manifest. Keyed by content sha.
dockerfile(text/plain)— SENSITIVE/PRIVATE: a content-addressed (sha256) Dockerfile emitted by lock_env (FROM mambaorg/micromamba, micromamba install -f environment.yml). Env-independent — one shared file across a project's envs. Keyed by content sha.
nix(text/plain)— SENSITIVE/PRIVATE: a content-addressed (sha256) Nix flake SCAFFOLD emitted by lock_env — best-effort attrs + a pinned nixpkgs ref, honestly labeled NOT a reproducing lock (verify attrs; bioconda/python packages need fallbacks). Keyed by content sha.
03Reports
@vcjdeboer/capture-reportmethod
capture_report.ts
Render a captured CS session (session-ingest facets) into a readable report: the narrative arc (research question -> plan -> conclusion), the tools/packages/skills the session loaded, the independent reviewer's detail, your verbatim prompts, and a few captured figures embedded inline. Reads only sealed swamp data.
capturesession-ingestprovenance
04Previous Versions
2026.07.21.1yanked
2026.07.15.2yanked
Modified 1 models
2026.07.15.1yanked
2026.07.12.1yanked
Modified 1 models
2026.07.11.17yanked
Modified 1 models
2026.07.11.16yanked
2026.07.11.15yanked
2026.07.11.14yanked
2026.07.11.13yanked
Modified 1 models
2026.07.11.12yanked
Added 1 reports
2026.07.11.11yanked
Modified 1 models
2026.07.11.10yanked
2026.07.11.9yanked
2026.07.11.8yanked
2026.07.11.7yanked
2026.07.11.6yanked
Modified 1 models
2026.07.08.1yanked
05Stats
A
100 / 100
Downloads
0
Archive size
268.7 KB
- Has README or module doc2/2earned
- README has a code example1/1earned
- README is substantive1/1earned
- Most symbols documented1/1earned
- No slow types (deprecated)1/1earned
- Dependencies pass trust audit2/2earned
- Has description1/1earned
- Platform support declared (or universal)2/2earned
- License declared1/1earned
- Verified public repository2/2earned
06Platforms
07Labels