Skip to main content
← Back to list
01Issue
FeatureOpenSwamp CLIPublic
AssigneesNone

Relationships

#4963 data query --select reads only the content paths the expression needs

Opened by zack · 11/20/2026

Problem

swamp data query ... --select '{phase: content.phase}' returns a tiny projection, but each matching JSON record is still read whole and parsed with JSON.parse (fromRow / parseContent in src/domain/data/data_record_mapper.ts, and DataQueryService.projectedContents). Query memory scales with the full size of every matched record, not with what the select reads. The same applies to queryData from behaviors, which goes through the same service.

Proposal

  • Work out from the select's CEL AST which content paths it reads: member chains rooted at content and its attributes alias, static field and index access, and the comprehension macros narrowed to receiver[*] plus the iteration variable's chains. Anything not recognised falls back to the whole document.
  • When the select reads content and the filter does not, parse each record with the streaming JSON projector from swamp-club#4942, keeping only those paths.
  • A pruned record must never reach an output that shows the full record.

Expected outcome

Select results are identical to today. Memory is proportional to what the select reads. A property test shows that a select over the pruned document equals the select over the full document.

Context

Split out of swamp-club#4942. Depends on the streaming JSON parser and the projection types delivered there.

02Bog Flow
◉OPEN○TRIAGED○IN PROGRESS○SHIPPED

Open

11/20/2026, 7:35:00 PM

No activity in this phase yet.

03Sludge Pulse

Sign in to post a ripple.