JavaScript Web Scraping Tools

DuckDB

In-process SQL analytics, no server needed

Infrastructure · Self-hosted

DuckDB is an in-process analytical database that lets you query scraped CSV, Parquet, and JSON files with fast SQL, no server required.

Analytics Databases

Polars

Lightning-Fast DataFrame Library in Rust

Library or tool · Self-hosted

Polars is an open-source DataFrame library with a Rust-based query engine, columnar processing, lazy evaluation, and parallel execution.

Data Transformation & Validation

Milvus

Open-source vector database for billion-scale AI

Infrastructure · Self-hosted

Milvus is an open-source vector database that handles billion-scale embeddings with GPU acceleration for AI-powered search and retrieval.

Vector Databases

ClickHouse

Real-time analytics database for petabyte-scale data

Infrastructure · Self-hosted

ClickHouse is a fast, open-source columnar database that runs analytical queries on billions of scraped records in milliseconds.

Analytics Databases

Meilisearch

Lightning-fast search engine through configured exporters or custom item pipelines

Infrastructure · Self-hosted

Meilisearch is a fast, typo-tolerant search engine that indexes scraped data and delivers instant search-as-you-type results.

Search Engines

MinIO

Archived community S3-compatible object store

Infrastructure · Self-hosted

MinIO is a self-hosted, S3-compatible object store for files, datasets, and media, with distributed storage and erasure coding. Its repository is archived.

Object Storage

Redis

In-memory data store for caching and queues

Infrastructure · Self-hosted

Redis is a blazing-fast in-memory data store used to cache scraped results, manage job queues, and power real-time data pipelines.

Caching Databases

Elasticsearch

Distributed RESTful search and analytics engine

Infrastructure · Self-hosted

Stars: 78.3K
Code license: AGPL-3.0-only OR SSPL-1.0 OR Elastic-2.0 (default source headers)

Elasticsearch is a distributed search and analytics engine that lets you index, query, and analyze large volumes of scraped data in real time.

Search Engines

Playwright

Cross-Browser Automation for Modern Web Apps

Library or tool · Library / Self-hosted

Stars: 97.8K
Code license: Apache-2.0

Playwright is an open-source framework for reliable end-to-end testing and browser automation across Chromium, Firefox, and WebKit with a single API.

Browser Automation

+1 more

Puppeteer

JavaScript API for Chrome and Firefox Automation

Library or tool · Library / Self-hosted

Stars: 95.7K
Code license: Apache-2.0

Puppeteer is a JavaScript library that provides a high-level API to control Chrome or Firefox for web scraping, testing, and browser automation tasks.

Browser Automation

+1 more

bisibility

Growing a website? Track its keywords, Google rankings, Search Console insights, and share of voice with bisibility. Free during beta.

Try bisibility freeOur product

JavaScript Web Scraping Tools

Discover web scraping tools and libraries for JavaScript. Compare features, community support, and documentation.