Skip to main content
← back to extensions

Bench Datastore

@webframp/bench-datastorev2026.08.28.1· 13d agoMODELSWORKFLOWS
01README

Datastore benchmarking harness for swamp. Two workflow-based test scenarios that exercise any swamp datastore backend under sustained load without lock contention.

Scenario A: Throughput (breadth)

Each worker owns isolated models. Cycles through write → read → query operations measuring per-step latency. Tests: sync push/pull speed, chunked storage, GSI queries, workflow overhead.

Scenario B: Write Stress (depth)

Each worker owns one model, writes continuously with varying payload sizes. Tests: sustained single-writer throughput, version accumulation, chunked vs inline performance, hot-partition behavior.

Usage

swamp extension pull @webframp/bench-datastore

# Create harness instance for worker 1 (throughput scenario)
swamp model create @webframp/bench-datastore/harness bench-harness-w001 \
  --global scenario=throughput --global worker_id=1 --global models_per_worker=50

# Setup (creates worker-owned models)
swamp model method run bench-harness-w001 setup

# Run throughput cycle once
swamp workflow run @webframp/bench-throughput-cycle \
  --input harness_model=bench-harness-w001 \
  --input probe_model=bench-probe-w001 \
  --input iteration=1

# Run write-stress once
swamp workflow run @webframp/bench-write-stress \
  --input harness_model=bench-harness-w001 \
  --input probe_model=bench-probe-w001 \
  --input iteration=1

Both workflows are designed to be called in a tight loop by an ECS Fargate worker or local script. Duration is controlled by the caller.

02Release Notes

2026.08.28.1

Changed: Normalized the extension license to Apache-2.0 and corrected the copyright holder to "Sean Escriva". Extensions that previously shipped an MIT LICENSE.md are now Apache-2.0, consistent with the repository root and every other extension. No code or behavioral changes.

Upgrade note: License text only. No API, schema, or runtime behavior changed.

2026.08.27.1

Changed: Test suite now builds its method context from the shared createModelTestContext factory in @systeminit/swamp-testing, replacing the per-test inline contexts with a single makeContext helper that wraps writeResource to preserve the existing assertion API. The local spawn()-aware withMockedCommand is retained. No behavioral, schema, or method changes — the published harness is unchanged. The deno.json gains a dev-only @systeminit/swamp-testing import-map entry.

2026.08.26.3

Fixed: Restored inline npm:zod@4.4.3 import specifiers so the registry quality scorer can resolve dependencies and score the extension. An earlier release used a bare "zod" import-map specifier, which published but scored as unscored.

Changed: Retained explicit compilerOptions.strict in deno.json. No behavioral or schema changes.

03Models1
@webframp/bench-datastore/harnessv2026.08.28.1harness.ts
fn setup()
Create the required models for this worker's scenario. Idempotent — safe to re-run.
fn execute()
Run a single iteration of the configured scenario. Returns timing data.

Resources

setup(infinite)— Setup result — models created for this worker.
result(infinite)— Single iteration execution result with timing data.
04Workflows2
@webframp/bench-throughput-cycled1a2b3c4-e5f6-7890-abcd-ef1234567890

Single iteration of the throughput benchmark. Executes one model method (rotated by iteration), then reads back the data and checks datastore health. Designed to be called in a tight loop by an ECS worker. Measures: write latency, read latency, datastore health, and workflow orchestration overhead.

writeExecute a model method (write operation with sync push).
1.run-method${{ inputs.harness_model }}.execute— Run the harness execute method for one throughput iteration.
readRead back data to measure sync pull latency.
1.data-search${{ inputs.probe_model }}.execute— Search recent data scoped to the harness model.
healthDatastore health check (round-trip latency).
1.status${{ inputs.probe_model }}.execute— Check datastore connectivity and latency.
@webframp/bench-write-stressa9b8c7d6-e5f4-4210-aedc-ba0987654321

Single iteration of the write-stress benchmark. Writes a payload of configured size to the worker's model, then reads it back to verify. Designed to be called in a tight loop with no sleep between iterations. Payload sizes rotate: small (100B), medium (10KB), large (500KB+). Large payloads exercise the chunked storage path. Measures: write throughput, chunk handling, version accumulation, read-after-write latency, sustained single-writer performance.

writeWrite a payload to the worker's single model.
1.execute-write${{ inputs.harness_model }}.execute— Run the harness execute method for one write-stress iteration.
verifyRead back the written data to confirm integrity.
1.read-back${{ inputs.probe_model }}.execute— Search recent data scoped to the harness model to verify write.
05Previous Versions8
2026.08.27.1

2026.08.27.1

Changed: Test suite now builds its method context from the shared createModelTestContext factory in @systeminit/swamp-testing, replacing the per-test inline contexts with a single makeContext helper that wraps writeResource to preserve the existing assertion API. The local spawn()-aware withMockedCommand is retained. No behavioral, schema, or method changes — the published harness is unchanged. The deno.json gains a dev-only @systeminit/swamp-testing import-map entry.

2026.08.26.3

Fixed: Restored inline npm:zod@4.4.3 import specifiers so the registry quality scorer can resolve dependencies and score the extension. An earlier release used a bare "zod" import-map specifier, which published but scored as unscored.

Changed: Retained explicit compilerOptions.strict in deno.json. No behavioral or schema changes.

2026.08.26.3

2026.08.26.3

Fixed: Restored inline npm:zod@4.4.3 import specifiers so the registry quality scorer can resolve dependencies and score the extension. An earlier release used a bare "zod" import-map specifier, which published but scored as unscored.

Changed: Retained explicit compilerOptions.strict in deno.json. No behavioral or schema changes.

2026.08.26.1

2026.08.26.1

Fixed: Added missing description field to upgrade entry for version 2026.08.24.2. The omission caused swamp extension pull to fail with a catalog validation error ("upgrades.N.description: Invalid input: expected string, received undefined").

2026.08.24.3

2026.08.24.3

Added: Output metadata attributes for observability.

  • durationMs: Method execution duration in milliseconds.
  • collectedBy: Extension name that produced the data.
  • fetchedAt: ISO 8601 timestamp when data was fetched (added to resources that previously lacked it).
2026.08.24.2

2026.08.24.2

Fixed development section to use standard deno commands. Added Troubleshooting section documenting intentional no-retry design, setup idempotency, subprocess timeout absence, correct --global-arg flag, and external datastore selection.

2026.08.21.1

2026.08.21.1

Changed: When a benchmark write iteration fails, the recorded errorMessage now names the target model and payload size instead of surfacing the bare stderr from the inner swamp data write command. A command that failed with no stderr output previously produced an empty error message; it now says so explicitly.

2026.07.27.1

Fixed: The fmt task ran deno fmt --check, so deno task fmt verified formatting instead of applying it and there was no way to format the extension through its own task. fmt now formats and a new fmt:check verifies, matching every other extension in the repository.

Fixed: deno fmt no longer inspects CLAUDE.md / AGENTS.md. Those files are gitignored and never present in CI, but deno fmt does not read .gitignore, so deno task fmt:check could fail locally on a file CI does not have.

Upgrade note: Tooling and formatting only. No model, method, schema, or behavior change — nothing to do on upgrade.

2026.07.27.1

2026.07.27.1

Fixed: The fmt task ran deno fmt --check, so deno task fmt verified formatting instead of applying it and there was no way to format the extension through its own task. fmt now formats and a new fmt:check verifies, matching every other extension in the repository.

Fixed: deno fmt no longer inspects CLAUDE.md / AGENTS.md. Those files are gitignored and never present in CI, but deno fmt does not read .gitignore, so deno task fmt:check could fail locally on a file CI does not have.

Upgrade note: Tooling and formatting only. No model, method, schema, or behavior change — nothing to do on upgrade.

2026.07.24.1

2026.07.24.1

Added: Initial release of the datastore benchmarking harness.

  • Two workflow-based scenarios: throughput (breadth) and write-stress (depth)
  • Zero lock contention by construction — each worker owns isolated models
  • Backend-agnostic measurement of datastore I/O independent of swamp core locking
  • Configurable at runtime via workflow inputs (scenario, worker count, duration controlled by caller)
  • Deployable on ECS Fargate or runnable locally
  • Payload delivery uses stdin streaming via temp files to avoid Linux MAX_ARG_STRLEN (131072 bytes) limit on large (500KB) write-stress payloads
  • Probe measurements in workflows are scoped to the worker's harness model via --model filter to prevent cross-worker data inflation under concurrent load
  • payload_size accepted as an optional workflow input on bench-write-stress for controlled single-size runs

Added 1, removed 1, modified 1 workflows

06Stats
A
100 / 100
Downloads
0
Archive size
16.9 KB
  • Has README or module doc2/2earned
  • README has a code example1/1earned
  • README is substantive1/1earned
  • Most symbols documented1/1earned
  • No slow types (deprecated)1/1earned
  • Dependencies pass trust audit2/2earned
  • Has description1/1earned
  • Platform support declared (or universal)2/2earned
  • License declared1/1earned
  • Verified public repository2/2earned
Repository
https://github.com/webframp/swamp-extensions
07Platforms
08Labels
#bench#datastore#load-test#performance