Run self-recovering queue workers in Convex by providing a work query and worker mutation, with debouncing, polling, and automatic restart built in.
npm install @convex-dev/batch-workerBatch Worker provides a durable, self-recovering loop for processing queued work stored in your own Convex tables. You supply a work query that returns the next batch and a worker mutation that processes it, and the component handles scheduling, debouncing, single-loop enforcement, snapshot reads to avoid OCC conflicts, and automatic restart on failure. It is the underlying pattern behind components like Workpool, exposed so you can build custom queue processors with full control over your data model.
Batch Worker splits queue processing into two steps: a snapshot-read work query that scans the queue without creating read dependencies, and a worker mutation that receives the fetched batch as args. This means concurrent inserts do not conflict with the fetch scan, eliminating OCC retries. You call ping() after inserting work, and the component ensures exactly one loop runs per named worker.
Batch Worker lets you insert incremental update rows into a dedicated table instead of writing directly to a shared aggregate document. A named worker drains that table in batches, applying updates to the aggregate in sequence. This decouples high-concurrency writers from the single aggregate writer, avoiding database conflicts at the cost of slight staleness.
Because the worker mutation is a transaction, it cannot call external APIs directly. The pattern is to have the mutation claim a batch by marking rows as in-progress, then schedule an action or enqueue it in a Workpool. The action performs the external call and commits results via a follow-up mutation. Batch Worker can return debounceMs from the mutation to throttle how fast batches are assembled.
Batch Worker integrates with the Convex Rate Limiter component. The worker mutation can use the rate limiter's reserve functionality to obtain a retryAfter value, then return that value as debounceMs to delay the next batch. Alternatively, the work query can call rateLimiter.check() and return idle with a timeoutMs to defer execution until capacity is available.
No. Batch Worker does not insert, delete, or modify your work rows. Your worker mutation is responsible for deleting or advancing past the rows it processes. If the mutation does not clean up processed rows, the next work query will return them again.
Batch Worker monitors the running loop and automatically restarts it if it stops unexpectedly. The failure is logged so you can set up alerting on it. This recovery happens without any manual intervention or external cron job.
Yes. You call ping() with a distinct name for each queue. The component runs exactly one loop per named worker. Your work query receives the name as args.name, so a single query and mutation pair can serve multiple queues by branching on that value.
There are two debounce points. The debounceMs in the ping() config delays the first batch after waking from idle, allowing a burst of inserts to accumulate before processing starts. The debounceMs returned from the worker mutation delays the next iteration while the loop is already running, which is useful for throttling throughput. Pings received during a debounce are not lost: the loop always re-runs the work query when the debounce elapses.
Convex's built-in scheduler and crons can trigger functions on a time basis but do not manage a work queue, handle concurrency guarantees for a single running loop, or restart automatically on failure. Batch Worker is specifically designed to drain a table-backed queue with exactly-one-loop semantics, debouncing, cooldown polling, and self-recovery. It is the pattern underlying the Workpool component.