BullMQ
BullMQ is a task queue for background jobs. Its JavaScript/TypeScript package supports Redis; version 6.3.11 also includes an optional PostgreSQL backend. Backend support must be checked for each language binding.
BullMQ runs background tasks through queues and workers. It supports scheduling, retries and job dependencies.
The JavaScript/TypeScript package supports Redis and includes an optional PostgreSQL backend in version 6.3.11. PostgreSQL uses the optional pg driver and an explicit backend configuration. Check the documentation for the language binding and version you use; backend support should not be assumed to be identical across bindings.
The open-source package is MIT licensed. BullMQ Pro is a separate commercial offering.
Web scraping workflow
How to use BullMQ in a web scraping pipeline
BullMQ distributes JavaScript scraping jobs between queue producers and workers. Its Queue and Worker model lets an application accept collection requests without making the user wait for the actual fetch and parsing work.
A practical workflow
- Add a job containing a normalized URL, crawl identifier and extraction version to a queue.
- A Node.js worker calls your HTTP client or Playwright routine, validates the extracted fields and writes the result under a stable record key.
- Record the result location and completion state for the application; route failed jobs into an operational review process.
When it fits
Choose this pattern when a Node.js service needs background scraping with independently operated worker processes. Small job payloads and explicit crawl identifiers make it easier to inspect which collection request produced a record.
Boundaries
You still supply the scraper, queue infrastructure and destination database. Worker concurrency is not a substitute for per-domain pacing. Design result writes to tolerate repeated processing, and do not treat a queue completion event as evidence that a page contained valid business data.
Growing a website? Track its keywords, Google rankings, Search Console insights, and share of voice with bisibility. Free during beta.
Our productRepository
taskforcesh/bullmq
Stars
9.5KForks
720Last commit
2026-10-15License
MITLanguage
TypeScript
Metrics recorded 2026-10-16
Repository details
License source: Repository license file
Frequently asked questions
What is BullMQ used for?
BullMQ queues background jobs for workers to process, with retries, delayed execution, scheduling and job dependencies. Redis is its default backend; the BullMQ 6.3.11 JavaScript/TypeScript package also includes an optional PostgreSQL backend. Your application supplies the work performed by each job.
Sources (checked ): BullMQ guide; BullMQ PostgreSQL backend; Published BullMQ 6.3.11 npm metadata; PostgreSQL factory at npm gitHead
Can BullMQ be used with PostgreSQL?
Yes. The BullMQ 6.3.11 JavaScript/TypeScript package includes an optional PostgreSQL backend selected with createPostgresBackend. It requires PostgreSQL 13 or newer and the pg package. Redis remains the default backend. Verify backend support when using a different package version.
Sources (checked ): BullMQ PostgreSQL backend; Published BullMQ 6.3.11 npm metadata; PostgreSQL factory at npm gitHead; PostgreSQL minimum version at npm gitHead
Is BullMQ free?
BullMQ is MIT licensed. You still provide its database backend and worker infrastructure. BullMQ Pro adds separately licensed commercial features, so distinguish the core package from Pro and from hosting costs.
Sources (checked ): BullMQ repository; BullMQ Pro
Is BullMQ available on npm?
Yes. The JavaScript/TypeScript package is published as bullmq and the official examples install it with npm. Configure a supported backend and worker process after installing the package.
Sources (checked ): BullMQ repository; BullMQ guide
Is there a BullMQ library for Python?
Yes. BullMQ documents a Python package installed with pip install bullmq. Its examples use asyncio and a Redis connection. Check the Python API and supported features separately from the JavaScript/TypeScript package.
Sources (checked ): BullMQ Python introduction
What are the key differences between RabbitMQ and BullMQ?
BullMQ is a job-queue library with worker APIs and database backends. RabbitMQ is a message broker with queueing, routing and messaging protocols. Compare whether you need application job management or broker-level messaging and how you will operate each system.
Sources (checked ): BullMQ guide; BullMQ PostgreSQL backend; RabbitMQ documentation
Is BullMQ similar to Kafka?
Both can participate in asynchronous processing, but BullMQ manages jobs for workers while Kafka provides an event-streaming platform with retained logs and consumer processing. Compare retry, scheduling, replay and messaging requirements rather than treating them as interchangeable.
Sources (checked ): BullMQ guide; Apache Kafka introduction
What are some alternatives to BullMQ?
RabbitMQ can serve broker-based queueing needs; Inngest and Temporal provide durable workflow execution with different programming and deployment models. Compare the job lifecycle and infrastructure responsibilities needed by your application.
Sources (checked ): BullMQ guide; RabbitMQ documentation; Inngest documentation; Temporal documentation
Sources
Verified against the taskforcesh/bullmq repository and the npm package.