Relationships
#922 /lab loads every issue with full markdown bodies on every request (incl. detail pages) + ~12 serial DB waves — the 3.4s avg / 30s tail
Opened by keeb · 7/2/2026· Shipped 7/2/2026
From the 2026-07-02 full-stack performance audit. /lab pages average 3.4s, max 30s (hits the app's own 30s middleware timeout → 504). The ~700ms middleware floor is tracked separately; this is the additional ~2.7s+, which is multiplicative across the findings below. GitHub lab auto-sync is disabled (main.ts:38-43,286-301) and is NOT the cause.
1. Every /lab request fetches every issue ever filed, full markdown bodies included
routes/lab/index.tsx:76, routes/lab/[slug].tsx:172,281 → mongo-lab-issue-repository.ts:532-599. Every board render, every filter view, and every issue detail page calls repo.list({visibilityFilter}) with no limit and no projection — full docs including the entire body of every issue, plus a parallel countDocuments.
Compounding:
- No usable index: only
{status,createdAt},{type,createdAt},{source,createdAt}exist (:335-372); the visibility$and-of-$ors query with.sort({createdAt:-1})is a collscan + in-memory sort of every doc, bodies included. - The full array is serialized twice per view — SSR HTML + LabShell island hydration props.
- On
/lab/{number}detail, the full board load happens in addition to the detail queries (sidebar list).
This is the growth-shaped defect matching "3.4s avg, 30s max": cross-provider transfer of N full markdown bodies per request.
Fix: (a) {body: 0} projection for list cards (ship a bodyExcerpt if client search needs text, or move search server-side); (b) createIndex({createdAt: -1}); (c) cap + paginate; (d) on detail routes, drop or lazily fetch the board (the SPA API routes/api/v1/lab/issues/[number].ts:94 already returns detail without it).
2. Detail handler: ~12 serial DB waves that could be ~4
routes/lab/[slug].tsx:123-235: loadViewerCollectiveIds → findByNumber → 6-query Promise.all → enrichAuthorPlans → resolveMentions → resolveTiers → findByIssueIds, each await gating the next. ~600ms-1s of pure serial latency; mentions/tiers/relationships/enrichment are mutually independent after the main wave. Same shape in the filter-view branch (:280-307) and routes/lab/index.tsx:57-104 (~8 waves → ~4).
Fix: restructure into ~4 Promise.all waves.
3. listUserCollectives / listCollectivesForUsers: 3 serial round trips each, twice per request
better-auth-collective-queries.ts:98-124 and :142-181: member find → await → organization find → await → organization_profile find. Runs on every /lab page for logged-in viewers (viewer-collectives.ts:25) and again inside enrichAuthorPlans on cache miss. The docstring calls it "a single batched read" — it's 3 serial trips. 6 serial trips → 2-3 per cold /lab request.
Fix: single aggregation ($lookup chain), or at minimum fetch organization + organization_profile concurrently.
4. enrichAuthorPlans cold cache: findOne per collective (#804)
lib/app/lab/enrich-author-plans.ts:112-117 → mongo-subscription-repository.ts:49-52. With finding 1 putting ALL authors on every page, this is 100-300 concurrent findOnes on cache miss — pool pressure that inflates every other in-flight request. Already tracked as #804 (findByCollectiveIds batch method); noting here that it's on the /lab hot path and part of this stack.
5. listCommentedIssueIds: collscan of all lab comments per authenticated request
mongo-lab-issue-repository.ts:732-737: comments.find({authorId}, {projection:{issueId:1}}) — no authorId index exists (:342-346), and it's unbounded. Runs on every logged-in board/detail/filter request.
Fix: createIndex({authorId: 1, issueId: 1}) or comments.distinct("issueId", {authorId}).
6. Per-request markdown render + mention colorize of entire threads, uncached
routes/lab/[slug].tsx:228-232,252-261 (same in routes/api/v1/lab/issues/[number].ts:165-196): GFM render + linkify + per-username regex colorize for body + every comment (cap 200) + lifecycle entries, per GET. Immutable content, pure recompute; CPU stacks on the event loop under concurrency.
Fix: bounded LRU keyed (sourceId, updatedAt, tier-map-hash), or store bodyHtml at write time and re-render on edit.
7. /lab/insights listCreatedSince collscan
mongo-lab-issue-repository.ts:880-886: find({createdAt: {$gte}}) — no single-field createdAt index, no projection (full bodies fetched; insights uses 4 fields). Fixed by finding 1's index + a projection.
Expected impact
3.4s → well under 500ms. Findings 1+2+3 alone plausibly account for the entire ~2.7s delta; finding 1 is the 30s-tail mechanism.
Shipped
Click a lifecycle step above to view its details.
Sign in to post a ripple.