Skip to main content

WORK OFFLINE AGAINST SWAMP SERVE

This guide shows you how to keep working on a behavior while the swamp serve your checkout writes through is unreachable, and how to land that work when you reconnect. For the rules, see Working Offline.

Prerequisites

  • A checkout whose v3 datastore is written through swamp serve (team mode). Any other checkout refuses the lease commands.
  • A grant to run the behavior's methods on serve.
  • Serve reachable now, and at least one command already run against it, so the checkout's copy has read serve's feed. A copy that never has refuses every command offline.

Take a lease while online

A method that changes things runs offline only on a behavior whose lease the checkout holds. Acquire one before you lose serve, for as long as you expect to be away (5s to 24h):

swamp behavior lease acquire my-server --for 8h
Lease held: my-server (aws/ec2)
  Lock:    data/aws/ec2/<behavior-id>/.lock
  Fence:   4
  Hold:    480m (renewed by each command in this checkout)
  Expires: <time>

Each command run online renews the lease. Offline, nothing renews it: it holds until the Expires time, by serve's clock. Take one lease per behavior you plan to change. See Behavior Leases for release and status.

Work offline

Run commands as usual. A command that starts while serve is unreachable warns that it works from the checkout's copy, and saves what it changes locally:

Serve is unreachable, so this work is saved in this checkout and will be sent to serve by the next command that reaches it

The command exits 0. Later commands, online or offline, read what earlier ones saved.

Work Offline
Reads, and read and list methods Run from the copy, which may be out of date
A method or workflow step on a behavior you leased Runs until the lease ends; a run is cancelled if it ends
A method or workflow step on any other behavior Refused, datastore_v3_offline_lease_required, exit 75
Editing or deleting a behavior, workflow or vault Refused, datastore_v3_offline_config_edit, exit 75

A method with no declared kind counts as one that changes things. A refused method changes nothing:

Serve is unreachable, so <method> on <behavior> cannot run: it changes things and needs the behavior's lease, which this checkout does not hold. Nothing was changed. To work offline, take the lease while online: swamp behavior lease acquire <behavior> --for <duration>.

A command that started online and loses serve partway fails with exit 75 and is not saved for later. Run it again.

Reconnect and read the flush report

Once serve is reachable, run any command that writes, such as a method run. It sends the saved work first, one entry at a time in the order saved. Read-only commands never send it.

The command reports what it sent:

Sent 2 change(s) this checkout saved while serve was unreachable
Versions saved offline were renumbered after what landed meanwhile: <type>/<id> <name> 3→5
<type>/<id> <name>: the latest version stays the one observed last, restated as version <n> (its content is version <m>'s)
  • Renumbered versions were moved after versions other writers saved while you were away.
  • Latest stays with the version observed last: your offline version becomes latest only when it ran more than 5 minutes after the current latest. See Latest after an offline rebase.
  • Waiting entries (<n> change(s) saved offline are still waiting in this checkout's outbox; a later command sends them) could not be sent yet: serve became unreachable again, or another holder's lease covers the behavior. Run a writing command again later.

With --json, the report is a datastore_offline_flush notice on stderr, with landed, waiting, rejected, quarantined, renumbered, latest and latestAtEnd.

Check for kept writes

List the entries the flush could not land:

swamp datastore status

They appear under Saved In This Checkout, each with its state, behavior, transaction id and reason:

Saved In This Checkout
  quarantined model--aws%2Fec2--web (22222222-2222-4222-8222-222222222222, saved 2026-10-30T12:01:00.000Z)
    Not authorized to write model:web (data.save)
  • rejected — serve could not apply the entry, for example because a version it deletes changed meanwhile. It is never sent again. Repeat the work if you still need it.
  • quarantined — serve refused it because you lack a grant it needs, for example one revoked while you were offline. Serve keeps it for an admin; a change larger than 32 MiB stays in the checkout only.

Accept or discard a quarantined write (admin)

An admin running swamp datastore status also sees the changes serve keeps, under Quarantined Changes:

Quarantined Changes
  held 33333333-3333-4333-8333-333333333333 - model--aws%2Fec2--web by user:bob
    Not authorized to write model:web (data.save)

Land a change as its author:

swamp datastore quarantine accept 33333333-3333-4333-8333-333333333333

Or drop it so it never lands:

swamp datastore quarantine discard 33333333-3333-4333-8333-333333333333

An accepted change is still checked against its author's current access to everything it references. See swamp datastore quarantine accept.

Release the lease

When you are done, end the lease so others can write the behavior:

swamp behavior lease release my-server

End an abandoned lease (admin)

A lease outlives a lost laptop or a crashed checkout until it expires, and survives a serve leader takeover. An admin ends it from any serve-mode checkout, naming the behavior as <type>/<name or id>:

swamp datastore lock release --force --behavior aws/ec2/my-server

Work its holder saved offline is then sent under no lease when that checkout reconnects, renumbered after what landed meanwhile.