News
01README
The full news stack — feed discovery, catalog management, preference-aware fetching, LLM story fusion, and static HTML rendering. Ships four model extensions (news-feed-discovery, news-feed-catalog, news-reader, news-feed-analysis), one report extension (news_html_report), the four news workflows (news-fetch, news-fusion, news-curation, news-full), and the decoupled feedback server that closes the 👍/👎 loop between the generated HTML page and the workflows. When the optional @svendowideit/systemd-service extension is installed, the feedback server runs as a systemd user service (via user lingering) so it starts at boot; otherwise the ensure-feedback-server step is skipped and you run it manually. Install: swamp extension pull @svendowideit/news. Model instances (local-news, feed-catalog, feed-discovery, feed-analysis) are auto-registered by the workflows on first run — no manual swamp model create needed. Setup: swamp model @svendowideit/news-reader method run setup local-news (set llmModel + llmBaseUrl to enable LLM fusion). Run: swamp workflow run @svendowideit/news-fetch | @svendowideit/news-fusion | @svendowideit/news-curation | @svendowideit/news-full.
02Models
@svendowideit/news-readerv2026.09.10.1news_reader.ts
Global Arguments
| Argument | Type | Description |
|---|---|---|
| llmBaseUrl | string | Base URL of an OpenAI-compatible LLM server. The /v1/chat/completions path is used. Defaults to a local Ollama instance. |
| llmModel | string | LLM model tag used for story fusion (Ollama model, e.g. llama3, mistral, qwen2.5). |
| llmApiKey? | string | Optional API key for LLM servers that require authentication. Ollama typically does not need one. |
| llmTemperature | number | Sampling temperature for fusion LLM calls. Low values keep extraction deterministic. |
| fusionMinClusterSize | number | Minimum cluster size to trigger LLM fusion. Singletons stay as ordinary summaries unless a story already exists. |
| citationRetentionDays | number | How long article citations live before aging out of a story. Core facts always survive. |
| llmConcurrency | number | Max concurrent LLM requests for fusion steps. Start at 1 and increase if the server supports it. |
| maxFusions | number | Hard cap on the number of LLM fusion calls made within a single fusion step (fuseStories / seedStories / regenStories). Once reached, the remaining clusters are skipped and persisted as-is. |
| llmTimeoutSec | number | Per-call timeout (seconds) for LLM chat/completions requests. Lower values surface server outages faster. |
| llmFailureThreshold | number | Server-side LLM failures (outage / timeout / HTTP 5xx) tolerated in a step before the step halts. 1 = fail fast on the first server error. Client-side errors are not counted. |
| feedbackServerPort | number | Port the feedback queue HTTP server listens on (default 8765). |
| feedbackServerServiceName | string | systemd user service name for the feedback server. |
| feedbackServerScriptPath? | string | Override the path to the bundled feedback-server.ts script. Defaults to the script bundled with this extension. |
fn setup(llmBaseUrl?: string, llmModel?: string, llmApiKey?: string, llmTemperature?: number, fusionMinClusterSize?: number, citationRetentionDays?: number, llmConcurrency?: number, maxFusions?: number, llmTimeoutSec?: number, llmFailureThreshold?: number)
Interactive configuration helper. Run with no inputs to list the model's config params (name, kind, default, current value); pass inputs to validate them, test the LLM settings live, and print the exact `swamp model edit` command to persist only the changed values. NOTE: setup does NOT persist values itself.
| Argument | Type | Description |
|---|---|---|
| llmBaseUrl? | string | Base URL of an OpenAI-compatible LLM server (e.g. http://localhost:11434) |
| llmModel? | string | LLM model tag for story fusion. Empty = fusion disabled; set a tag to enable. |
| llmApiKey? | string | Optional API key for LLM servers that require authentication. |
| llmTemperature? | number | Sampling temperature (0..2). Low values keep extraction deterministic. |
| fusionMinClusterSize? | number | Minimum cluster size to trigger LLM fusion. |
| citationRetentionDays? | number | How long article citations live before aging out. |
| llmConcurrency? | number | Max parallel LLM requests. |
| maxFusions? | number | Hard cap on LLM fusion calls per step before the step stops and the workflow continues. |
| llmTimeoutSec? | number | Per-call LLM request timeout in seconds. |
| llmFailureThreshold? | number | Server-side LLM failures tolerated in a step before the step halts. |
fn cleanupCdata()
Strip CDATA wrappers from existing keywords in snapshots and preferences. Run once to clean up data from before the CDATA-stripping fix.
fn fetch(feeds: array, maxArticlesPerFeed: number)
Fetch RSS/Atom feeds and store articles
| Argument | Type | Description |
|---|---|---|
| feeds | array | RSS/Atom feed URLs to fetch — either string URLs or feed objects from news-feed-catalog |
| maxArticlesPerFeed | number | Maximum articles to keep per feed (default 25) |
fn dedupeArticles()
Group articles by URL, mark duplicates, and annotate primary articles with duplicate source info
fn filterByAge(newsAge: string)
Filter articles by age and store filtered snapshot for HTML generation
| Argument | Type | Description |
|---|---|---|
| newsAge | string | Time range of news to show (e.g., 2h, 7d, 4w, 1m). Supports h (hours), d (days), w (weeks), m (months). Defaults to 3 days. |
fn generate(topN: number, title: string)
Generate static HTML report from latest articles, ranked by interest
| Argument | Type | Description |
|---|---|---|
| topN | number | Number of articles to include in the HTML report (0 = all articles, default 0) |
| title | string | Title for the HTML report page |
fn generateMobile(topN: number, title: string)
Generate the mobile/tablet-optimised news summary page (swipe pagination + in-page article reader iframe) from the latest articles, ranked by interest. Writes to `~/.swamp/news-pages/news-mobile.html` by default.
| Argument | Type | Description |
|---|---|---|
| topN | number | Number of articles to include in the mobile HTML report (0 = all articles, default 0) |
| title | string | Title for the mobile HTML report page |
fn feedback(articleId: string, action: enum, source?: string, title?: string, keywords?: array)
Record user feedback on an article (interested or ignored)
| Argument | Type | Description |
|---|---|---|
| articleId | string | Article ID (hash of URL) |
| action | enum | Whether the user found this article interesting or ignored it |
| source? | string | Feed source name |
| title? | string | Article title |
| keywords? | array | Keywords/tags from the article |
fn gatherFeedback(serverUrl: string, batchSize: number, maxBatches: number)
Poll the feedback queue HTTP server, import entries into preferences, and delete processed entries
| Argument | Type | Description |
|---|---|---|
| serverUrl | string | URL of the feedback queue HTTP server |
| batchSize | number | Number of feedback entries to process per batch |
| maxBatches | number | Maximum number of batches to process in one run |
fn gatherPages(serverUrl: string, batchSize: number, maxBatches: number, category?: string)
Poll the pages queue HTTP server, return queued page URLs as a data resource
| Argument | Type | Description |
|---|---|---|
| serverUrl | string | URL of the pages queue HTTP server |
| batchSize | number | Number of queued pages to process per batch |
| maxBatches | number | Maximum number of batches to process in one run |
| category? | string | Category tag applied to each page when adding to the catalog |
fn clusterArticles(snapshotName: string, maxClusterSize: number)
Conservatively cluster filtered articles into same-story groups (no LLM). Requires a filtered-snapshot resource.
| Argument | Type | Description |
|---|---|---|
| snapshotName | string | |
| maxClusterSize | number |
fn seedStories(minClusterSize: number)
LLM-seed persistent Story objects from the current clusters resource.
| Argument | Type | Description |
|---|---|---|
| minClusterSize | number |
fn fuseStories()
LLM-delta pass: absorb new cluster articles into existing stories and persist.
fn regenStories()
Throttled P3 pass: full LLM re-fusion of each story from its citations.
fn renderStories()
Render persistent stories to an inline HTML fragment resource (storiesHtml) and optionally a standalone stories.html page.
fn ensureFeedbackServer(port?: number, serviceName?: string, scriptPath?: string)
Idempotently ensure the feedback queue HTTP server is running as a systemd user service. Resolves the bundled feedback-server.ts script path, creates the unit via @svendowideit/systemd-service (if that extension is installed), and starts it. Skips gracefully (with a log) if the systemd-service extension is not installed.
| Argument | Type | Description |
|---|---|---|
| port? | number | Port the feedback server listens on (defaults to global feedbackServerPort). |
| serviceName? | string | systemd user service name (defaults to global feedbackServerServiceName). |
| scriptPath? | string | Override the path to the feedback-server.ts script (defaults to the bundled script). |
Resources
snapshot(7d)— Latest fetched feed snapshot
filteredSnapshot(7d)— Age-filtered snapshot for HTML generation
preferences(infinite)— User article preferences and learned keyword weights
pagesQueue(7d)— Page URLs pulled from the pages queue for catalog upsert
stories(infinite)— Persistent fused story objects (survive age-filter windows)
clusters(7d)— Transient story clusters awaiting LLM seed/fuse
storiesHtml(7d)— Rendered inline HTML fragment for fused stories
dedupedUrls(7d)— Previously deduped article URLs for incremental dedupe
Files
report(text/html)— Static HTML news summary page
@svendowideit/news-feed-catalogv2026.08.06.1784424438feed_catalog.ts
Global Arguments
| Argument | Type | Description |
|---|---|---|
| catalogName | string | Catalog name (for multiple catalogs) |
| dedupeStalenessDays | number | Max days before a cached feed identity is re-fetched during dedupe. New feeds are always fetched. |
fn add(url: string, category: string, name?: string)
Add a feed URL to the catalog
| Argument | Type | Description |
|---|---|---|
| url | string | RSS/Atom feed URL to add |
| category | string | Category tag (e.g., tech, news, programming, podcasting) |
| name? | string | Human-readable feed name (defaults to hostname) |
fn seed(url: string, category: string)
Ensure the catalog has at least one feed. If the catalog is empty (or missing), add the default swamp-club feed so the news stack has something to fetch on a fresh install. No-op when the catalog already has feeds.
| Argument | Type | Description |
|---|---|---|
| url | string | Feed URL to add when the catalog is empty |
| category | string | Category tag for the seeded feed |
fn dedupe()
Fetch each catalog feed, group duplicates by content identity, and mark the less expressive (or second) feed as a duplicate.
fn generateFeedsHtml(title: string, prefs?: object, snapshot?: object)
Generate a static HTML page listing all feeds in the catalog, grouping each feed with its duplicates.
| Argument | Type | Description |
|---|---|---|
| title | string | Page title |
| prefs? | object | News-reader preferences used for per-feed seen/read/interested/ignored counts |
| snapshot? | object | News-reader feed snapshot used to map article IDs to feed sources |
fn gatherFeedState(serverUrl: string, batchSize: number, maxBatches: number)
Poll the feedback server's /api/feed endpoint, apply enabled/disabled state to the catalog, and delete processed entries.
| Argument | Type | Description |
|---|---|---|
| serverUrl | string | URL of the feedback queue HTTP server |
| batchSize | number | Number of feed state entries to process per batch |
| maxBatches | number | Maximum number of batches to process in one run |
fn remove(url: string)
Remove a feed URL from the catalog
| Argument | Type | Description |
|---|---|---|
| url | string | RSS/Atom feed URL to remove |
fn list(category?: string, limit: number)
List feeds in the catalog. If category is omitted, all categories are returned. Use limit=0 for unlimited.
| Argument | Type | Description |
|---|---|---|
| category? | string | Filter by category (optional) |
| limit | number | Maximum feeds to return. Use 0 for unlimited. |
fn listCategories()
List all unique categories in the catalog
Resources
catalog(infinite)— Feed catalog (list of RSS/Atom feeds with categories)
categories(infinite)— List of unique categories in the catalog
dedupe(infinite)— Result of the dedupe method
dedupe-cache(infinite)— Cached feed identity data for incremental deduplication
Files
report(text/html)— Generated static HTML pages (e.g. the feeds listing)
@svendowideit/news-feed-discoveryv2026.08.03.1feed_discovery.ts
Global Arguments
| Argument | Type | Description |
|---|---|---|
| newsReaderModelId | string | Model ID of the news-reader instance to read snapshots from (empty = find any) |
| feedCatalogModelId | string | Model ID of the news-feed-catalog instance to upsert into (empty = find any) |
fn discover(maxSitesToCrawl: number, category: string, dryRun: boolean, reCrawlAfterDays: number, articleUrls?: array, nonFeedUrls?: array)
Discover RSS/Atom feeds from article domains, skipping recently-crawled sites and prioritising the most promising new domains
| Argument | Type | Description |
|---|---|---|
| maxSitesToCrawl | number | Maximum new domains to crawl this run (default 10) |
| category | string | Category tag for discovered feeds (default 'discovered') |
| dryRun | boolean | If true, discover feeds but don't add them to the catalog (default false) |
| reCrawlAfterDays | number | Domains crawled within this many days are skipped unless their feed is still unknown (default 7) |
| articleUrls? | array | Article URLs to crawl for new feeds (strings or {url} objects). When |
| nonFeedUrls? | array | URLs known to be HTML pages (not feeds) to crawl (strings or {url} |
Resources
discovery(30d)— Result of the most recent feed discovery run
crawlLedger(infinite)— Ledger of domains already crawled for feed discovery
@svendowideit/news-feed-analysisv2026.08.03.1feed_analysis.ts
Global Arguments
| Argument | Type | Description |
|---|---|---|
| newsReaderModelId | string | Model ID of the news-reader instance whose pages-current data to read |
fn analyzePages(maxPagesPerRun: number, probeCommonPaths: boolean, pages?: array)
Analyze pages gathered by the news-reader, discover RSS/Atom feeds in each page, and write a page-discovery-result resource
| Argument | Type | Description |
|---|---|---|
| maxPagesPerRun | number | Maximum pages to analyze this run (default 50) |
| probeCommonPaths | boolean | Probe common feed paths on pages that advertise no feed link (default true) |
| pages? | array | Page entries gathered by the news-reader's gatherPages method. When |
Resources
pageDiscovery(30d)— Result of the most recent page-analysis run
03Workflows
@svendowideit/news-fetch22d55bc4-a25e-48af-a0ba-6c2cef52f761
Fast-path news workflow — gathers feedback, fetches feeds from the catalog, deduplicates articles, filters by age, and generates the HTML report. Runs every 4 hours. No LLM calls, no catalog maintenance, no feed discovery. Fusion (LLM story clustering) runs separately via the @svendowideit/news-fusion workflow. Curation (feed dedupe, discovery, page analysis) runs separately via the @svendowideit/news-curation workflow. Loop: gather-feedback → fetch → dedupe-articles → filter → generate → gene
mainGather feedback, fetch feeds, filter by age, generate HTML.
1.gather-feedback— Poll the feedback queue server and import any pending feedback entries into preferences.
2.ensure-feedback-server— Idempotently ensure the feedback queue HTTP server runs as a systemd user service (via @svendowideit/systemd-service). Skips with a log if that extension is not installed; no-ops if the service is already running.
3.seed— Ensure the catalog has at least one feed (adds the default swamp-club feed when empty) so a fresh install has something to fetch.
4.fetch— Fetch RSS feeds from the news-feed-catalog (or from --input feeds if provided).
5.dedupe-articles— Group articles by URL, mark duplicates, and annotate primary articles with duplicate source info so the filter step skips them and the HTML shows cross-feed indicators.
6.filter— Filter articles by age (default: last 3 days).
7.generate— Generate HTML report from filtered articles, ranked by preferences, with fused stories rendered inline (from fusion workflow).
8.generate-mobile— Generate the swipe-driven mobile/tablet news page (news-mobile.html).
9.generate-feeds-html— Render a static feeds.html listing all catalog feeds, grouped by duplicates, with per-feed seen/read/interested/ignored article counts and a cross-link back to the news summary.
@svendowideit/news-fusionf5d98a8b-5f6e-45f9-a2f6-2938dbb16545
LLM-driven story fusion — clusters filtered articles into same-story groups, fuses them into persistent stories via LLM delta passes, seeds new stories for fresh clusters, and renders stories to an inline HTML fragment. Runs every 12 hours. All steps allow failure — fusion is best-effort enrichment. The news workflow reads the rendered stories HTML from the datastore and includes it in the news page. Steps: cluster → fuse → seed → render
mainCluster articles, fuse into stories, seed new stories, render HTML.
1.cluster— Conservatively group filtered articles into same-story clusters (no LLM).
2.fuse— LLM-delta pass — absorb clustered articles into existing persistent stories.
3.seed— LLM-seed persistent Story objects for fresh clusters with no existing story.
4.render— Render persistent stories to an inline storiesHtml fragment and a standalone stories.html page.
@svendowideit/news-curation27ae2513-825f-4445-b325-97cbb40ed816
Feed catalog maintenance — deduplicates feeds by content identity, gathers feed enable/disable state from the feedback server, discovers new RSS feeds from article URLs, and analyzes queued pages for feed discovery. Runs daily at 3am. All steps allow failure — curation is best-effort maintenance. The news workflow reads the catalog's current state directly from the datastore. Steps: dedupe → gather-feed-state → discover → upsert-feed gather-pages → analyze-pages → upsert-page (independe
mainDeduplicate catalog, gather feed state, discover new feeds, analyze pages.
1.seed— Ensure the catalog has at least one feed (adds the default swamp-club feed when empty) so a fresh install has something to fetch.
2.dedupe— Fetch each catalog feed, group by content identity, and mark duplicate feeds so fetch skips them.
3.gather-feed-state— Poll the feedback server's /api/feed endpoint, apply enabled/disabled state to the catalog, and delete processed entries.
4.discover— Discover new RSS feeds from article URLs.
5.upsert-feed-${{ self.feed.url }}— Add each discovered feed to the news-feed-catalog.
6.gather-pages— Poll the pages queue server and collect queued page URLs into a data resource.
7.analyze-pages— Analyze each gathered page and discover RSS/Atom feeds, writing a page-discovery-result resource.
8.upsert-page-${{ self.feed.url }}— Add each discovered page feed to the news-feed-catalog.
@svendowideit/news-fullf04794eb-33d9-443f-a582-3f12699c54e1
Full news feedback loop — gathers feedback from the queue server, fetches feeds from the catalog, generates an HTML summary ranked by your preferences, then discovers new RSS feeds from the article URLs and upserts them into the catalog for the next run, finally rendering feeds.html (a duplicate-grouped catalog listing). Loop: gather-feedback → dedupe (flag duplicate catalog feeds) → fetch → dedupe-articles (merge duplicate articles by URL) → filter by age (default: last 3 days) → cluster relat
mainGather feedback, fetch feeds, filter by age, generate HTML, discover new feeds for next run.
1.gather-feedback— Poll the feedback queue server and import any pending feedback entries into preferences.
2.seed-catalog— Ensure the catalog has at least one feed (adds the default swamp-club feed when empty) so a fresh install has something to fetch.
3.dedupe— Fetch each catalog feed, group by content identity, and mark duplicate feeds so fetch skips them.
4.gather-feed-state— Poll the feedback server's /api/feed endpoint, apply enabled/disabled state to the catalog, and delete processed entries.
5.fetch— Fetch RSS feeds from the news-feed-catalog (or from --input feeds if provided), skipping feeds marked duplicate by the dedupe step.
6.dedupe-articles— Group articles by URL, mark duplicates, and annotate primary articles with duplicate source info so the filter step skips them and the HTML shows cross-feed indicators.
7.filter— Filter articles by age (default: last 3 days).
8.cluster— Conservatively group filtered articles into same-story clusters (no LLM).
9.fuse— LLM-delta pass — absorb clustered articles into existing persistent stories.
10.seed— LLM-seed persistent Story objects for fresh clusters with no existing story.
11.render— Render persistent stories to an inline storiesHtml fragment and a standalone stories.html page.
12.generate— Generate HTML report from filtered articles, ranked by preferences, with fused stories rendered inline.
13.generate-mobile— Generate the swipe-driven mobile/tablet news page (news-mobile.html).
14.generate-feeds-html— Render a static feeds.html listing all catalog feeds, grouped by duplicates, with per-feed seen/read/interested/ignored article counts and a cross-link back to the news summary.
15.discover— Discover new RSS feeds from article URLs.
16.upsert-feed-${{ self.feed.url }}— Add each discovered feed to the news-feed-catalog.
17.gather-pages— Poll the pages queue server and collect queued page URLs into a data resource.
18.analyze-pages— Analyze each gathered page and discover RSS/Atom feeds, writing a page-discovery-result resource.
19.upsert-page-${{ self.feed.url }}— Add each discovered page feed to the news-feed-catalog.
04Reports
@svendowideit/news-html-reportmethod
news_html_report.ts
Summary of the generated news HTML page with article count
newssummary
05Previous Versions
2026.09.10.1
Modified 1 models
2026.08.30.1
Modified 1 models
2026.08.22.1
2026.08.20.1
Modified 1 models
2026.08.19.1
2026.08.18.1
2026.08.08.3
06Stats
A
100 / 100
Downloads
0
Archive size
184.1 KB
- Has README or module doc2/2earned
- README has a code example1/1earned
- README is substantive1/1earned
- Most symbols documented1/1earned
- No slow types (deprecated)1/1earned
- Dependencies pass trust audit2/2earned
- Has description1/1earned
- Platform support declared (or universal)2/2earned
- License declared1/1earned
- Verified public repository2/2earned
07Platforms