Skip to main content
← Back to list
01Issue
BugShippedSwamp Club
Assigneeskeeb

Relationships

#922 /lab loads every issue with full markdown bodies on every request (incl. detail pages) + ~12 serial DB waves — the 3.4s avg / 30s tail

Opened by keeb · 7/2/2026· Shipped 7/2/2026

From the 2026-07-02 full-stack performance audit. /lab pages average 3.4s, max 30s (hits the app's own 30s middleware timeout → 504). The ~700ms middleware floor is tracked separately; this is the additional ~2.7s+, which is multiplicative across the findings below. GitHub lab auto-sync is disabled (main.ts:38-43,286-301) and is NOT the cause.

1. Every /lab request fetches every issue ever filed, full markdown bodies included

routes/lab/index.tsx:76, routes/lab/[slug].tsx:172,281 → mongo-lab-issue-repository.ts:532-599. Every board render, every filter view, and every issue detail page calls repo.list({visibilityFilter}) with no limit and no projection — full docs including the entire body of every issue, plus a parallel countDocuments.

Compounding:

  • No usable index: only {status,createdAt}, {type,createdAt}, {source,createdAt} exist (:335-372); the visibility $and-of-$ors query with .sort({createdAt:-1}) is a collscan + in-memory sort of every doc, bodies included.
  • The full array is serialized twice per view — SSR HTML + LabShell island hydration props.
  • On /lab/{number} detail, the full board load happens in addition to the detail queries (sidebar list).

This is the growth-shaped defect matching "3.4s avg, 30s max": cross-provider transfer of N full markdown bodies per request.

Fix: (a) {body: 0} projection for list cards (ship a bodyExcerpt if client search needs text, or move search server-side); (b) createIndex({createdAt: -1}); (c) cap + paginate; (d) on detail routes, drop or lazily fetch the board (the SPA API routes/api/v1/lab/issues/[number].ts:94 already returns detail without it).

2. Detail handler: ~12 serial DB waves that could be ~4

routes/lab/[slug].tsx:123-235: loadViewerCollectiveIds → findByNumber → 6-query Promise.all → enrichAuthorPlans → resolveMentions → resolveTiers → findByIssueIds, each await gating the next. ~600ms-1s of pure serial latency; mentions/tiers/relationships/enrichment are mutually independent after the main wave. Same shape in the filter-view branch (:280-307) and routes/lab/index.tsx:57-104 (~8 waves → ~4).

Fix: restructure into ~4 Promise.all waves.

3. listUserCollectives / listCollectivesForUsers: 3 serial round trips each, twice per request

better-auth-collective-queries.ts:98-124 and :142-181: member find → await → organization find → await → organization_profile find. Runs on every /lab page for logged-in viewers (viewer-collectives.ts:25) and again inside enrichAuthorPlans on cache miss. The docstring calls it "a single batched read" — it's 3 serial trips. 6 serial trips → 2-3 per cold /lab request.

Fix: single aggregation ($lookup chain), or at minimum fetch organization + organization_profile concurrently.

4. enrichAuthorPlans cold cache: findOne per collective (#804)

lib/app/lab/enrich-author-plans.ts:112-117 → mongo-subscription-repository.ts:49-52. With finding 1 putting ALL authors on every page, this is 100-300 concurrent findOnes on cache miss — pool pressure that inflates every other in-flight request. Already tracked as #804 (findByCollectiveIds batch method); noting here that it's on the /lab hot path and part of this stack.

5. listCommentedIssueIds: collscan of all lab comments per authenticated request

mongo-lab-issue-repository.ts:732-737: comments.find({authorId}, {projection:{issueId:1}}) — no authorId index exists (:342-346), and it's unbounded. Runs on every logged-in board/detail/filter request.

Fix: createIndex({authorId: 1, issueId: 1}) or comments.distinct("issueId", {authorId}).

6. Per-request markdown render + mention colorize of entire threads, uncached

routes/lab/[slug].tsx:228-232,252-261 (same in routes/api/v1/lab/issues/[number].ts:165-196): GFM render + linkify + per-username regex colorize for body + every comment (cap 200) + lifecycle entries, per GET. Immutable content, pure recompute; CPU stacks on the event loop under concurrency.

Fix: bounded LRU keyed (sourceId, updatedAt, tier-map-hash), or store bodyHtml at write time and re-render on edit.

7. /lab/insights listCreatedSince collscan

mongo-lab-issue-repository.ts:880-886: find({createdAt: {$gte}}) — no single-field createdAt index, no projection (full bodies fetched; insights uses 4 fields). Fixed by finding 1's index + a projection.

Expected impact

3.4s → well under 500ms. Findings 1+2+3 alone plausibly account for the entire ~2.7s delta; finding 1 is the 30s-tail mechanism.

02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPED+ 1 MOREASSIGNED+ 4 MOREREVIEW+ 4 MOREPR_MERGED+ 1 MORENOTIFICATION_SKIPPED

Shipped

7/2/2026, 11:15:29 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
keeb assigned keeb7/2/2026, 9:29:43 PM

Sign in to post a ripple.