Batching & Parallel Processing

Use batching when you need to process a large list of items in parallel while controlling concurrency. Items are split into fixed-size batches, each batch runs concurrently, and failures in one batch don't affect others.

When to use this

  • Bulk data imports (contacts, orders, or products from a comma-separated values (CSV) file)
  • Processing hundreds or thousands of items against external APIs
  • Calling rate-limited APIs where you need to control concurrency
  • Any fan-out where you want failure isolation between groups
  • High-concurrency fan-out, where one flat Promise.all over the whole list would put more work in flight than your downstream services or the run's event log should carry at once

How it works

  1. Records are split into fixed-size batches.
  2. Each batch runs in parallel through Promise.allSettled, so failures in one record don't affect others.
  3. A sleep() between batches paces requests to avoid overloading downstream services.
  4. After all batches, a summary is returned with succeeded/failed counts.

Pattern

The workflow splits records into chunks, processes each chunk concurrently, tracks results per batch, and returns a final tally.

import { sleep } from "workflow";

type Record = { name: string; email: string; role: string };

declare function processRecord(record: Record): Promise<string>; // @setup

export async function batchImport(records: Record[], batchSize: number) {
  "use workflow";

  let totalSucceeded = 0;
  let totalFailed = 0;

  for (let i = 0; i < records.length; i += batchSize) {
    const batch = records.slice(i, i + batchSize);

    // Run batch in parallel: failures are isolated per record
    const outcomes = await Promise.allSettled( 
      batch.map((record) => processRecord(record))
    );

    for (let j = 0; j < outcomes.length; j++) {
      if (outcomes[j].status === "fulfilled") {
        totalSucceeded++;
      } else {
        totalFailed++;
      }
    }

    // Pace between batches to avoid overloading downstream
    if (i + batchSize < records.length) {
      await sleep("1s");
    }
  }

  return { total: records.length, succeeded: totalSucceeded, failed: totalFailed };
}

Step function

Each record is processed in its own step with full Node.js access and automatic retries.

type Record = { name: string; email: string; role: string };

async function processRecord(record: Record): Promise<string> {
  "use step";
  const res = await fetch(`https://api.example.com/contacts`, {
    method: "POST",
    body: JSON.stringify(record),
  });
  if (!res.ok) throw new Error(`Failed to import ${record.email}`);
  const { id } = await res.json();
  return id;
}

Adapting to your use case

  • Replace the Record type with your actual data shape, such as orders, images, or products.
  • Replace processRecord() with your import logic, such as database upserts, API calls, or file processing.
  • Tune batchSize and the sleep() duration to match your downstream rate limits.
  • Add or remove tracking as needed; the pattern works with any item type.

Tips

  • Use Promise.allSettled instead of Promise.all: Use this pattern when you want to continue even if some items fail. Promise.all rejects on the first failure, while allSettled waits for everything and identifies failures.
  • Tune batch size to your downstream API limits: If the API allows 10 concurrent requests, use batchSize: 10.
  • Batching bounds concurrency, not the run's total size: Every batch still appends to the same event log, so a long enough list walks one run toward its run limits no matter how small the batches are. To shrink the run itself, bundle more items per step so one step covers many items, or spawn a child workflow per batch. Reach for one of those once a single run would grow past a few thousand events.
  • Add pacing with sleep(): Add a delay between batches to respect rate limits. The sleep is durable and survives cold starts.
  • Treat each processRecord call as an independent step: If one call fails, it retries up to three times without affecting other items in the batch.

Key APIs