Relationships
#4213 @swamp/agent-sessions: explore a decision model (TypeSafe Jev) for call labelling
Opened by skunk-ape · 10/31/2026
Follow-up to swamp-club#4157, which labels every imported call swamp / bypass / neutral and computes findings with rules only (shell segment parsing, swamp-managed paths, coverage from installed extensions by a naming convention). Repo: swamp-extensions, agent-sessions/.
Explore using a decision model, such as TypeSafe's System One model Jev (wrapped by @swamp/typesafe-ai), to do or assist the classification. Jev does not generate text: it takes state plus typed questions and returns calibrated answers (Noul yes/no probability, Choice over a fixed option set with a distribution, Score along an ordered rubric). That shape fits labelling: a Choice over swamp / bypass / neutral, or over categories, from a call's tool, kind and redacted target.
Questions to answer:
- Accuracy against the rules on the three #4157 fixture sessions (status-checker, track-star, gatorwalk) and on harder calls the rules get wrong or call opaque: scripts, unclear writes, programs an extension covers that the naming convention misses.
- Rules first with the model only for low-confidence or unknown calls, or the model throughout?
- Determinism and explainability: each label must still record what decided it; how a calibrated probability shows up as the rule.
- Privacy: only redacted call text may leave the machine; never prompts, replies or output. Is redacted text enough for a good answer?
- Cost and latency at volume (about 9k sessions and 70k calls a month for a heavy user), batching via ask, and caching by call text.
- Where it runs: inside the importer at import time, or as a separate relabel method over stored records (stored text is redacted and has lost heredoc bodies).
Out of scope until decided: changing the shipped rules or the record contract.
Open
No activity in this phase yet.
Sign in to post a ripple.