Research Collector
Research data collector — gathers intelligence from Hacker News, Lobste.rs, arXiv, SRE Weekly, IFIN Discourse, RedMonk, and The AI Daily Brief for daily AI/tech intelligence briefings. Produces typed resources for downstream workflows.
Usage
swamp model create @webframp/research-collector research-collector
# Gather all sources
swamp model method run research-collector gather
# Build a compact digest from the latest brief
swamp model method run research-collector digestMethods
- gather — Gather data from HN front page, Lobste.rs hottest, arXiv, SRE Weekly, IFIN Discourse security topics, RedMonk, and The AI Daily Brief.
- digest — Build a compact digest from the latest brief: top items per source, cross-source topic clusters, and a delta against the previous digest. Run gather first.
2026.07.23.1
Added: New digest method builds a compact daily digest from the latest
research brief: top items per source (normalized 0-100 prominence score),
cross-source topic clusters (keywords appearing in 2+ sources), and a delta
against the previous digest (new vs carried items). Writes a new digest
resource (24h lifetime) for downstream journal/briefing workflows.
Changed: The model now reads prior resources via the readResource context
method. digest requires a gather run first — it throws if no brief exists.
Upgrade note: No changes to the research brief schema or existing global
args. Existing instances upgrade in place (no attribute migration needed). The
new digest resource is additive and safe to ignore for workflows that only
use gather.
Global Arguments
| Argument | Type | Description |
|---|---|---|
| hnCount | number | Number of Hacker News front-page stories to fetch |
| lobstersCount | number | Number of Lobste.rs hottest stories to fetch |
| sreCount | number | Number of SRE Weekly issues to fetch |
| ifinCount | number | Number of IFIN Discourse topics to fetch |
| redmonkCount | number | Number of RedMonk blog posts to fetch |
| arxivCount | number | Number of arXiv paper entries to fetch |
| aiDailyBriefDays | number | Number of recent The AI Daily Brief editions to gather (one per day) |
Resources
2026.07.21.1
Added: The AI Daily Brief as a seventh gather source. gather now fetches
recent editions from theaidailybrief.com, keeps written takeaway "nuggets" with
headings, bodies, and anchors, and drops video embeds so only written analysis
survives. Controlled by a new aiDailyBriefDays global arg (1-14, default 3).
Fixed: The test module had unescaped quotes in three XML fixture string literals that stopped Deno from parsing the file, so the AI Daily Brief coverage test never ran. Switched those fixtures to template literals.
Changed: gather's output resource now carries an aiDailyBrief object
with editions alongside the existing source arrays. Existing sources and
resource shapes are unchanged.
Add The AI Daily Brief as a seventh source: scrapes daily written editions from aidailybrief.ai, keeps written analysis nuggets, discards video/audio embeds. New global arg aiDailyBriefDays (1-14, default 3).
2026.07.18.1
Added: An upgrades array entry (no-op) to research_collector.ts for proper typeVersion tracking on existing instances. No schema or behavior changes.
- Has README or module doc2/2earned
- README has a code example1/1earned
- README is substantive1/1earned
- Most symbols documented1/1earned
- No slow types (deprecated)1/1earned
- Dependencies pass trust audit2/2earned
- Has description1/1earned
- Platform support declared (or universal)2/2earned
- License declared1/1earned
- Verified public repository2/2earned