Events are leased, not held
A worker claims an event with a lease. If it finishes, the event is completed and its consequences are queued before the claim is released — so there is never a moment where the queue looks empty while work is still in flight. If the worker dies, the lease expires and another worker picks the event up. Delivery is at-least-once, and the engine says so plainly rather than implying exactly-once.
claim · lease · reclaimA gather’s arrivals are rows
The obvious implementation of a fan-in appends to an array on one record. That record becomes a contention point, and under a forty-way fan-out the losers of each write conflict lose their items silently. Here each arrival is its own row at its own id, and completion is claimed by taking the rows: exactly one worker gets them, the rest stand down.
one row per arrival, never an arrayA join completes once, on whichever node
Branches of a join arrive on different workers at the same instant, by design. The arrival that completes it is decided by a conditional claim rather than by a read-and-check, because a read-and-check is the same race one layer up. This is where the engine has been wrong before, and both bugs are pinned by tests that run ten branches finishing together, five times over.
count, then claimWaiting costs nothing
An approval, an inbound signal, a delay, an interval: each is a row with a status or a time on it. No worker is parked, no thread is held, no memory is pinned. A run can wait days for a person and survive every replica restarting in the meantime — and the answer can land on whichever replica the person happens to reach.
wf_wait · wf_timerScaling out is running more of them
Every replica runs the same loop against the same database. There is no coordinator to elect and no shard to assign: they compete for events, and the claim decides. A schedule’s occurrence is claimed the same way, so four replicas do not mean four runs at 07:00.
shared queue, no leaderSecrets are not in the graph
A connector node names a credential; the credential holds the value, sealed, and no endpoint returns it — not to the UI, not to the API, not to an agent. That is what lets the same graph be exported, reviewed in a pull request and imported into another deployment without a secret travelling with it.
AES-256-GCM at restEvery run is a document, and it stays one
A run keeps the graph as it was when it started, its per-task status, repetition count, last result and error, its event ledger, and its log with the published widgets replayed into it. Records a graph wrote carry the run id, and artifacts carry the run that filed them — so the figure on a published page is traceable to the execution that produced it.
the run recordRoles and tokens
Roles are checked next to the route handlers, mirrored in the UI, never only in the UI. Scoped djwf_ tokens for scripts, CI and agents are minted and revoked per consumer, and an allow-list (WORKFLOW_HTTP_ALLOWLIST) bounds where a graph may call, when you want one.
admin · user · viewer · djwf_Limit: at-least-once delivery
A node that dies between doing its work and recording it will run again. Idempotency is therefore the author’s job where it matters, and the catalog’s connector nodes are written to make it easy — upserts rather than inserts, named artifacts rather than appended ones.
stated, not hiddenLimit: the database is the floor
Throughput is bounded by SurrealDB, because every event, arrival, wait and timer is a write. That is a deliberate trade: the failure modes it removes are worth more than the ceiling it imposes for the work this engine is for. It is not the right engine for a million events a second.
stated, not hiddenLimit: sandboxed, not isolated
A JavaScript node runs in a Bun worker with a timeout, which contains a mistake and an infinite loop. It is not a security boundary against a hostile author, and the deployment’s own controls — who may write a graph — are what stand in that place. Local file nodes are confined to a configured root, and disabled rather than unrestricted when that root is unset.
stated, not hidden