--- title: Eager processing of steps and incremental event replay description: Combine workflow event replay and step bundles to do work inline where possible, only deferring to the queue for parallelism. type: overview --- # Eager processing of steps and incremental event replay # Eager processing of steps and incremental event replay **Date**: March 2026 This is a major internal architecture change to how Workflow DevKit executes workflows and steps. It reduces function invocations and queue overhead by executing steps *inline* within the same function invocation as the workflow replay, rather than dispatching every step to a separate function via the queue. ## Previous architecture The previous architecture used two separate routes, each backed by its own queue trigger: ```text Queue: __wkf_workflow_* --> /.well-known/workflow/v1/flow (workflow replay in VM) | suspension (step needed) | queue step to __wkf_step_* | Queue: __wkf_step_* --> /.well-known/workflow/v1/step (step execution in Node.js) | step completes | queue continuation to __wkf_workflow_* v (cycle repeats for each step) ``` Each step required **2 queue messages** (step invocation and workflow continuation) and **2 function invocations**, plus cold-start overhead for each. A serial workflow with 10 steps needed approximately 21 function invocations. ## New architecture The two routes are merged into a single handler at `/.well-known/workflow/v1/flow` using `workflowEntrypoint()`. The step route is no longer generated. The handler runs an inline execution loop: ```text receive queue message | +-- if message has stepId+stepName: execute that step, queue workflow continuation, exit | v replay workflow in VM | +-- workflow completed --> create run_completed event, exit +-- workflow failed --> create run_failed event, exit | v suspension with pending operations | +-- process hooks and waits (unchanged) | +-- 0 pending steps --> return (waits/hooks only) +-- 1 pending step --> execute inline, loop back to replay +-- N pending steps --> queue N-1 to self (with stepId), | execute 1 inline, loop back to replay | +-- timeout check: if wall-clock time >= threshold, | re-schedule self via queue and exit | v (loop continues until completion, timeout, or non-step suspension) ``` A serial workflow with 10 steps now completes in **1 function invocation**. ## Background steps (parallel execution) When a workflow suspends with multiple pending steps (for example, from `Promise.all`), the handler creates `step_created` events for all of them, queues N-1 back to `__wkf_workflow_*` with `stepId` and `stepName` in the message payload, executes 1 step inline, and loops back to replay. Each background step message is handled by a separate function invocation of the same handler. When a message arrives with `stepId` and `stepName`, the handler executes that specific step, then checks if all parallel steps from the batch are done by comparing `step_created` events against terminal events (`step_completed`/`step_failed`): * **All steps done**: The handler replays the workflow inline, continuing the execution loop without a queue roundtrip. * **Steps still pending**: The handler returns without queuing a continuation. The last handler to complete its step will see all steps done and replay inline. * **Pending ops (stream writes)**: The handler queues a continuation and returns, so `waitUntil` can flush the pending stream data. ### Convergence after parallel steps When multiple background steps complete near-simultaneously, multiple handlers may observe "all steps done" and attempt to advance the workflow concurrently. The event-sourced architecture plus the invariants below ensure safe convergence: * **`step_created` idempotency**: Duplicate creates return 409; exactly one handler owns each step. * **`step_completed` / `step_failed` idempotency**: Only the first invocation to record a terminal result wins. * **Queue idempotency keys**: Background step messages use `correlationId` as the idempotency key. * **Deterministic replay**: All invocations produce the same result given the same event log. ### Single inline executor per step Inline step execution combined with background-step dispatch introduces a new coordination requirement: when multiple handlers reach the same `Promise.all` batch concurrently, we need to guarantee that each step body runs at most once via the inline path. Without that guarantee, the event log accumulates duplicate `step_started` events (including some written *after* `step_completed`, which orphans them on replay) and step bodies run redundantly. The design enforces one invariant: **exactly one handler owns each step, and only the owner may execute it inline**. Ownership is established by the atomicity of `step_created`: 1. **Atomic `step_created`**: `events.create('step_created', correlationId=X)` is serialized per correlation ID in every world. Exactly one concurrent caller succeeds; the rest receive `EntityConflictError`. 2. **Suspension handler reports ownership**: Only `step_created` writes that succeeded (not those that caught 409) count toward ownership. 3. **Inline execution is gated on ownership**: A handler that didn't win any `step_created` race performs no inline execution. 4. **Queueing is unconditional**: For every pending step except the one being executed inline, the handler enqueues a background step message with `idempotencyKey: correlationId`. This makes crash recovery work: if a prior handler wrote `step_created` but crashed before enqueueing, a later handler will enqueue the orphaned step. Concurrent handlers' redundant enqueues deduplicate on the idempotency key. Together these give: every `step_created` event has exactly one inline executor **and** at least one queued dispatch. Step bodies are never executed concurrently, and `step_started` events never land in the log after `step_completed` for the same step. **Retry semantics are preserved**: a step that is currently running (status=`running`) still accepts a second `step_started` write with an incremented attempt counter: this is how queue redelivery after a SIGKILL mid-execution legitimately re-runs the step. ## Incremental event loading The handler caches the event log in memory across loop iterations. Instead of re-fetching the entire event log on each replay: 1. **First iteration**: Load all events and return the final pagination cursor. 2. **Subsequent iterations**: Fetch only events created after the saved cursor and append them to the cached array. For a 10-step serial workflow completing in one invocation, the 10th replay loads \~2 new events instead of re-fetching all \~30. Incremental loading depends on the World returning a cursor even on the final page of results. If a World implementation does not return a cursor after the initial load, the handler logs an error and falls back to a full reload. ## Timeout handling The inline execution loop checks wall-clock time before each replay iteration. If the elapsed time exceeds a configurable threshold (110s by default for a 120-second function limit), the handler reschedules itself through the queue and returns. Configure the threshold with `WORKFLOW_V2_TIMEOUT_MS`. If a single step takes longer than the timeout threshold, the step runs to completion (or SIGKILL). There is no interruption mechanism for in-progress step execution. This is the same behavior as the previous architecture. ## Queue message changes The `WorkflowInvokePayload` schema has two new optional fields: `stepId` and `stepName`. When `stepId` is present, the handler executes that specific step before (or instead of) replaying the workflow. Background steps are queued with both set, so the handler knows which step function to call without loading the event log. Previously, `stepName` was resolved by loading all events and searching for the `step_created` event matching the `stepId`, an O(N) operation on the full event history for every background step arrival. The queue trigger configuration uses `WORKFLOW_QUEUE_TRIGGER` on the `__wkf_workflow_*` topic. The `__wkf_step_*` topic and its separate trigger are no longer generated. ## Generated file layout ```text .well-known/workflow/v1/ flow/ route.js # Handler (workflowEntrypoint) __step_registrations.js # Step function registrations (side effects) webhook/ [token]/ route.js # Webhook handler (unchanged) manifest.json # Workflow/step/class manifest (unchanged) config.json # Functions config (single trigger) ``` The `step/` directory is no longer generated. ## Design notes and tradeoffs ### Parent→child polling holds worker slots `Run#returnValue` is implemented as a polling step: the workflow awaits the child run's terminal status inside a step body. In worker-based Worlds (notably `world-postgres`), each such poll occupies a queue worker slot until the child run finishes. Parent workflows that fan out to many child runs, such as recursive workflows like `fibonacciWorkflow`, can therefore consume a large fraction of available workers while holding positions in `Promise.all([...children.map(c => c.returnValue)])`. If `queueConcurrency` is smaller than the peak number of concurrent parent polls plus the workers needed for any in-flight children, the system deadlocks: every slot is held by a parent waiting on a child, but no child can acquire a slot to start. For `world-postgres`, the default `queueConcurrency` is set to **50**. Workflows that fan out more aggressively must raise this ceiling. To prevent deadlock when polling is executed inline by the step executor, `Run#pollReturnValue()` detects when it's running inside a step executor and throws `TooEarlyError` instead of polling in a blocking loop. The step executor handles `TooEarlyError` by re-queueing the step with a 1-second delay, freeing the worker. Unlike `RetryableError`, `TooEarlyError` does NOT count against `maxRetries`, so polling steps can retry indefinitely until the child completes. **Follow-up**: Replace the worker-pool sizing requirement with a polling design that does not occupy a worker slot. Options under consideration: moving child-completion polling out of the step body into the suspension layer, or emitting a `run_completed` notification on the parent's stream/queue so the parent only resumes when the child actually finishes. ### Mixed suspensions A suspension may contain steps, hooks, and waits simultaneously. The handler creates events for all, then dispatches everything we are not running inline as a single parallel batch of queue messages: ```text ownedPendingSteps = pendingSteps.filter(owned by this handler) inlineStep = ownedPendingSteps[0] // optional dispatches = [ ...for each non-inline pendingStep: queue stepId message (idempotency=correlationId), ...if soonest pending wait: queue delayed continuation (delaySeconds=min(remaining, maxDelay), idempotency=waitCorrelationId[:hop|:secondBucket]), ] await Promise.all(dispatches) if (!inlineStep) return await executeStep(inlineStep) ``` The wait timer is queued as its own continuation rather than encoded in the handler's return value (`{ timeoutSeconds }`). This is what makes `Promise.race(step, sleep)` behave correctly: even when the inline step blocks the handler for the full step duration, the wait continuation fires in a separate function invocation. If the sleep wins, that parallel invocation observes `wait_completed` via the "complete elapsed waits" pass and finishes the run; if the step wins, the wait continuation fires later and no-ops on the terminal run via the existing terminal-event check. Step queueing remains unconditional (covers crash recovery: if a prior handler wrote `step_created` but crashed before queueing, a later handler will queue it; idempotency keys dedupe redundant queues across concurrent handlers). Wait continuations are likewise deduplicated, keyed on the wait's correlation ID: while a wait is pending, every replay pass over the run re-observes it and would otherwise enqueue another delayed continuation. A key is attached in all cases, since some worlds serialize key-less workflow messages per run, which would park the continuation behind the handler's own inline step execution. Two situations deliver a continuation while its wait is still pending, and each varies the key so the re-enqueue isn't dropped by a world's dedupe window (which outlives the first delivery): waits longer than the maximum queue delay (23h, bounded by VQS's 24h message retention) are clamped and chained across hops, with the hop index suffixed to the key so each hop dedupes within its window but the chain always advances; and near-elapsed waits (≤2s remaining) use a second-bucketed key suffix so a continuation delivered marginally early (clock skew) can enqueue a fresh short-delay retry. See `runtime/wait-continuation.ts` for the full selection logic. The retry/throttle and hook-conflict paths still return `{ timeoutSeconds }` since their semantics are "redeliver THIS message after a delay" rather than "schedule a fresh wait timer." Those can be unified in a follow-up. The unified dispatch requires `world-local` to honor `delaySeconds` on the queue (added in the same PR series). Without it, the wait continuation would fire instantly in dev and trigger a spurious replay before the wait elapsed (recoverable via redelivery, but inefficient and observable as duplicate `step_started` events under contention). ### VM sandboxing Workflow code still runs in a Node.js VM for determinism and sandboxing. Step code runs in the Node.js host context. The only change is that both happen within the same function invocation. ### Bundle size and cold start The combined bundle is larger (contains both step code and workflow VM code). Cold start time increases slightly. The reduction in total function invocations more than compensates. ### Step retries When an inline step fails with retries remaining: * `RetryableError` with explicit `retryAfter` delay: Requeue to self with `stepId` and a delay. * Transient errors with immediate retry: Requeue to self with `stepId` and a 1s delay. * `FatalError`: Fail immediately. ### Encryption key resolution Encryption keys are resolved once before the inline execution loop starts (after the run status is confirmed as `running`) and reused across all iterations. Background step executions resolve the key independently. The key does not change within a run. ### Module scope duplication in re-bundled output Builders that rebundle the combined output into a single file (standalone Workflow CLI, Vercel Build Output API, and NestJS) produce a layout where esbuild creates isolated module scopes for each source module, even within the same output file. Without intervention, `registerStepFunction` and `getStepFunction` operate on different `Map` instances: steps are registered into one `Map` but looked up from another. The step function registry and the step context storage are `globalThis` singletons (via `Symbol.for`) to ensure all module scopes share the same instances. The same pattern is used for the World singleton and the class serialization registry. ### Inline step execution with pending stream operations When a step's arguments or return value include serialized streams (e.g., `WritableStream` from `getWritable()`, or AI SDK streaming steps), the serialization layer creates background `flushablePipe` operations that pipe data to S3. These ops are tracked in an `ops` array and need to complete before the stream data is readable by external consumers. In V1, each step ran in a separate function invocation. After the step completed, `waitUntil(ops)` kept the function alive to flush the ops. In V2, the inline execution loop continues immediately after the step body returns, so we need to know whether to keep looping or break out and let `waitUntil` flush. `executeStep()` attempts a 500ms `Promise.race` between the ops settling and a timeout. If ops settle in time (data confirmed on server), it returns `hasPendingOps: false` and the V2 handler continues the inline loop. If ops don't settle in 500ms (e.g., `WritableStream` kept open across steps), it returns `hasPendingOps: true` and the V2 handler breaks the loop and queues a continuation so `waitUntil` can flush them. **Follow-up**: Shrink the 500ms inline-ops budget once we have confidence that the flush-waiter path settles deterministically across all worlds. A signaled "ops drained" event from the world layer would let `executeStep()` proceed without the timeout in the common case. ### Buffered stream flush with waiter promises `WorkflowServerWritableStream` buffers writes and flushes via a 10ms `setTimeout` for batching. Naively, `write()` could return immediately after buffering, but that would cause the `flushablePipe`'s `pendingOps` counter to reach 0 before data actually reached the server: the V2 inline loop would see ops as settled prematurely and produce data-loss races on every step with `WritableStream` serialization. `write()` returns a promise that resolves only after the scheduled flush completes. Multiple writes within the 10ms window still share a single batched HTTP request (the batching optimization is preserved). Each write registers a `{resolve, reject}` pair in a `flushWaiters` array. When the `setTimeout` fires and `flush()` completes the HTTP round-trip, all waiters are resolved (or rejected on error). This makes `pendingOps` accurately reflect server-side data state while keeping network-efficient batching. ### Lock-release polling interval `flushablePipe`'s `pollWritableLock` / `pollReadableLock` use `setInterval` to detect when a user releases their stream lock without closing the stream, since the Web Streams API has no event for that state. The V2 step executor's `opsSettled` race waits for this poll to resolve after each writable-bearing step body returns, so the polling interval sits on the critical path of every streaming step. The interval was lowered from 100ms to 10ms. Per-step wait drops from \~50ms average to \~5ms (scaling linearly with the number of writable-bearing steps in a workflow). Per-tick work is `writable.locked` plus a `getWriter()`/`releaseLock()` probe (microsecond-scale), so 10× more ticks is not measurable in practice. **Follow-up**: Replace polling entirely with an event-driven release signal (wrap the writable returned from the `WritableStream` reviver with a writer that fires on `releaseLock()`), bringing the wait to \~0ms. This would also remove a source of timing drift between worlds with synchronous storage (`world-local`) and worlds with HTTP-deferred storage (`world-vercel`). ### Concurrent `step_started` and attempt counter When the V2 handler dispatches N parallel steps as background messages, each background step completion queues a workflow continuation. Up to N continuations may replay concurrently, and each may attempt to start the same not-yet-completed step (since `step_started` succeeds for already-running steps). Each call atomically increments the `attempt` counter, so with N=5 parallel steps, the counter can reach 5 on the first genuine execution. The max retries check in `executeStep()` only enforces when `step.error` exists, distinguishing actual retries (failed → retry with error) from concurrent first-attempt races (multiple handlers start the same step simultaneously without any prior failure). Concurrent starts are harmless since `step_completed` idempotency ensures only the first completion wins. ### Unconsumed event check two-phase drain The `EventsConsumer`'s unconsumed event check uses a two-phase promise queue drain: yield once after the first drain (via `setTimeout(0)`) so cross-VM promise chains can append follow-up async work, then re-drain before checking. This improves timing for scenarios like `step_completed` → for-await loop resume → next hook hydration. The check additionally arms a `DEFERRED_CHECK_DELAY_MS = 100` `setTimeout` after the second drain, since Node.js does not guarantee that `setTimeout(0)` fires after all cross-context microtasks settle. Any `subscribe()` call arriving during that 100ms window cancels the check via version invalidation + `clearTimeout`, so the delay only adds latency to genuine corruption, never to the happy path. **Follow-up**: 100ms is a heuristic chosen empirically. A deterministic settlement signal (for example, a "VM idle" callback exposed by the workflow VM bridge that fires only after all pending cross-context promise chains have resolved) would let the consumer fire the unconsumed-event check immediately on quiescence instead of waiting for a wall-clock timeout. ### Lazy World loading Static imports of `world-local` and `world-vercel` from the runtime caused two distinct build-time issues: Next.js production builds pulled both worlds (including their Node-only deps like `debug`'s `tty` requires) into the route module, and Turbopack's NFT (Node File Trace) errored on `process.cwd()` and dynamic `import()` patterns it couldn't statically analyze. A `getWorldLazy()` accessor (backed by a `globalThis` `Symbol.for` cache) replaces the static import in step-side modules. This breaks the static import chain from step code to `world.ts`, preventing both worlds from being bundled into the step registrations. Because tree-shaking can otherwise drop `world.ts`'s module-load registration entirely, a server-only side-effect module (`@workflow/core/runtime/world-init`) imports `./world.js` purely for its module-load side effect. It's wired via package conditions: * `default` maps to the real module and loads `world.ts`. * `workflow` maps to an empty stub used by VM and step bundles. This guarantees the world is loaded for routes that consume `start()` without going through the queue-driven flow handler first, while keeping `world.ts` and its server-only deps out of the workflow sandbox bundle. ### Community Worlds and the `world.streams` API Community world adapters must implement the `world.streams.*` interface. The runtime legacy stream normalization was removed as part of this work. --- For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md) For an index of all available documentation, see [/llms.txt](/llms.txt) For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)