All tools

Every tool in the directory - scraping, anti-blocking, data processing, databases, infrastructure, and utilities.

ZenRows

AI-Ready Scraping With Anti-Bot Bypass

Platform or service · API

A commercial scraping API for bypassing bot defenses, rendering JavaScript, using residential proxies, and returning AI-ready results.

Scraping APIs

Stagehand

Automate Web Workflows With AI

Library or tool · Library / Self-hosted

Stars: 25.7K
Code license: MIT

Stagehand is an open-source SDK for AI-assisted browser automation with natural-language actions, structured extraction, and Playwright-style APIs.

AI Web Scraping

+1 more

Browserbase

Reliable Cloud Browsers for AI Agents

Platform or service · API

Run hosted browser sessions for AI automation and scraping with search, fetch, stealth, observability, and replay built in.

Headless Browsers

Browse AI

No-code web scraping and change monitoring

Platform or service · SaaS

Extract and monitor web data through recorded browser workflows, with spreadsheet exports and integrations.

AI Web Scraping

ParseHub

Visual scraper with free and paid plans

Platform or service

Extract data from dynamic websites by clicking elements, handle AJAX, forms, and dropdowns, then export to JSON, Excel, or API.

Web Scraping

Octoparse

Turn web pages into structured data, no code

Platform or service · Desktop app / SaaS / API

Build scrapers visually and turn web pages into structured data in minutes, with scheduling and an optional managed data service.

Web Scraping

Perplexity Sonar

Generative AI Search API with Real-Time Research

Platform or service · Legacy API · API

Perplexity Sonar provides models for web-grounded answers. Support for Sonar Chat Completions ended on September 27, 2026. Perplexity recommends Agent API for new integrations; existing clients should follow its migration guide.

Web Search APIs

ValueSERP

Low-Cost Google SERP API for Data Collection

Platform or service · API

ValueSERP provides Google search results through an API, with location targeting, batch processing, and JSON, HTML, or CSV output.

SERP Extraction APIs

SearchAPI.io

Multi-Engine Real-Time SERP API

Platform or service · API

SearchAPI.io delivers real-time SERP data from Google, Bing, Baidu, YouTube, and Amazon with structured JSON output.

SERP Extraction APIs

Zyte API

All-in-One Web Scraping API with AI Extraction

Platform or service · API

Zyte API is a full-stack web scraping API that combines anti-bot bypassing, browser rendering, and AI-powered data extraction in a single endpoint.

Scraping APIs

Exa AI

Semantic Web Search API Powered by AI

Platform or service · API

Exa AI is a modern search engine and web search API that uses semantic embeddings to deliver high-quality, real-time web data for AI applications.

Web Search APIs

Tavily

Real-Time Search API for AI Agents and RAG

Platform or service · API

Tavily is a real-time search engine for AI agents and RAG workflows, delivering fast web search and content extraction through a single secure API.

Web Search APIs

Brave Search API

Independent Web Search API with 35B+ Pages

Platform or service · API

Brave Search API provides enterprise-grade access to an independent index of 35+ billion web pages with real-time results optimized for AI and RAG pipelines.

Web Search APIs

Cloudflare R2

S3-compatible storage with zero egress fees

Infrastructure · API / SaaS

Cloudflare R2 is S3-compatible object storage with zero egress fees, making it ideal for storing and serving frequently accessed scraped data.

Object Storage

AWS Athena

Serverless SQL Queries on Amazon S3

Platform or service · API

Amazon Athena is a serverless, interactive query service that lets you analyze petabyte-scale data in S3 using standard SQL with pay-per-query pricing.

Data Processing

AWS Glue

Serverless Data Integration Service by AWS

Platform or service · API

AWS Glue is a fully managed, serverless ETL service that discovers, prepares, and integrates data from over 100 sources into lakes and warehouses.

ETL Tools

Webshare

Fast and affordable proxy servers

Platform or service · API

Webshare provides self-serve proxy servers across many locations, offering datacenter and residential proxy options for scraping workflows.

Proxy Services

IPRoyal

Premium proxies at affordable prices

Platform or service · API

IPRoyal offers residential, datacenter, ISP, and mobile proxy products with self-service access for scraping workflows.

Proxy Services

Decodo

AI-ready proxy and scraping solutions

Platform or service · API

Smartproxy (now Decodo) offers residential and datacenter proxy products with rotating and sticky sessions for scalable scraping.

Proxy Services

Render

Cloud platform for web services and cron jobs

Infrastructure · API / SaaS

Render is a cloud platform that deploys web services, background workers, and cron jobs with simple scaling for scraping workloads.

Cloud Compute

Railway

Deploy apps from GitHub through configured exporters or custom item pipelines

Infrastructure · API / SaaS

Railway deploys web apps and services from GitHub with auto-scaling, built-in databases, and instant previews for scraping workloads.

Cloud Compute

AWS Lambda

Serverless compute for event-driven pipelines

Infrastructure · API / SaaS

AWS Lambda runs code without managing servers, ideal for event-driven scraping pipelines triggered by schedules, queues, or API calls.

Cloud Compute

Cloudflare Workers

Run serverless code at the edge worldwide

Infrastructure · API / SaaS

Cloudflare Workers lets you run JavaScript and TypeScript at the edge with ultra-low latency for lightweight scrapers, proxies, and APIs.

Cloud Compute

ScraperAPI

Scale data collection with a simple API

Platform or service · API

ScraperAPI provides managed web scraping with proxy rotation, JavaScript rendering and CAPTCHA handling for supported public websites.

Scraping APIs

ScrapingBee

Web scraping API with proxy and browser handling

Platform or service · API

ScrapingBee is a web scraping API that handles headless browsers, rotates proxies, and offers AI-powered extraction so you can focus on your data.

Scraping APIs

DataForSEO

Comprehensive API Stack for SEO and Marketing Data

Platform or service · API

DataForSEO provides APIs for search engine results, keyword data, backlinks and other SEO and digital marketing datasets.

SERP Extraction APIs

CapSolver

AI-powered automatic CAPTCHA solving service

Platform or service · API

CapSolver is an AI-powered CAPTCHA solver that automatically bypasses reCAPTCHA, Cloudflare, AWS WAF, and other challenges for automation.

CAPTCHA Solvers

Oxylabs

Proxy services and web scraping APIs

Platform or service · API

Oxylabs provides proxy services, web scraping APIs and managed browser tools for collecting public web data.

Proxy Services

+1 more

SerpAPI

Real-Time Google Search Results API

Platform or service · API

SerpAPI is a real-time API that delivers structured Google search results, handling proxies, captchas, and rich data parsing automatically.

SERP Extraction APIs

Apify

Full-stack web scraping and data extraction platform

Platform or service · SaaS / API

Apify is a cloud platform for web scraping, browser automation, and data extraction with an Actor marketplace and developer SDKs.

Scraping APIs

BeautifulSoup

Python library for parsing HTML and XML

Library or tool · Library / Self-hosted

Beautiful Soup is a Python library for pulling data out of HTML and XML files, providing Pythonic idioms for navigating and searching parse trees.

HTML Parsers

Modal

Serverless Python platform for AI and data teams

Infrastructure · API / SaaS

Modal is a serverless Python platform that runs scraping, AI, and data workloads at scale with configurable containers and autoscaling.

Cloud Compute

2Captcha

CAPTCHA solving service with API integration

Platform or service · API

2Captcha is a CAPTCHA solving service that bypasses reCAPTCHA, Cloudflare Turnstile, hCaptcha, and other challenges via a simple API.

CAPTCHA Solvers

Zendriver

Async-first undetected browser automation

Library or tool · Self-hosted

Zendriver is a fast, async-first undetected browser automation framework based on nodriver with Docker support for scalable scraping.

Browser Automation

+1 more

Scrapy Cluster

Distributed on-demand scraping with Scrapy

Library or tool · Self-hosted

Scrapy Cluster uses Redis and Kafka to create a distributed, on-demand Scrapy crawling cluster for coordinated large-scale web scraping. Its repository is archived.

Large-Scale & Distributed Crawling

Frontera

Scalable crawl frontier framework

Library or tool · Self-hosted

Frontera is a Python crawl frontier framework for managing when and what to crawl, enabling web crawlers of any scale with Scrapy integration.

Large-Scale & Distributed Crawling

ZenRows

AI-Ready Scraping With Anti-Bot Bypass

Platform or service · API

A commercial scraping API for bypassing bot defenses, rendering JavaScript, using residential proxies, and returning AI-ready results.

Scraping APIs

Stagehand

Automate Web Workflows With AI

Library or tool · Library / Self-hosted

Stars: 25.7K
Code license: MIT

Stagehand is an open-source SDK for AI-assisted browser automation with natural-language actions, structured extraction, and Playwright-style APIs.

AI Web Scraping

+1 more

Browserbase

Reliable Cloud Browsers for AI Agents

Platform or service · API

Run hosted browser sessions for AI automation and scraping with search, fetch, stealth, observability, and replay built in.

Headless Browsers

Browse AI

No-code web scraping and change monitoring

Platform or service · SaaS

Extract and monitor web data through recorded browser workflows, with spreadsheet exports and integrations.

AI Web Scraping

ParseHub

Visual scraper with free and paid plans

Platform or service

Extract data from dynamic websites by clicking elements, handle AJAX, forms, and dropdowns, then export to JSON, Excel, or API.

Web Scraping

Octoparse

Turn web pages into structured data, no code

Platform or service · Desktop app / SaaS / API

Build scrapers visually and turn web pages into structured data in minutes, with scheduling and an optional managed data service.

Web Scraping

Perplexity Sonar

Generative AI Search API with Real-Time Research

Platform or service · Legacy API · API

Web Search APIs

ValueSERP

Low-Cost Google SERP API for Data Collection

Platform or service · API

ValueSERP provides Google search results through an API, with location targeting, batch processing, and JSON, HTML, or CSV output.

SERP Extraction APIs

SearchAPI.io

Multi-Engine Real-Time SERP API

Platform or service · API

SearchAPI.io delivers real-time SERP data from Google, Bing, Baidu, YouTube, and Amazon with structured JSON output.

SERP Extraction APIs

Zyte API

All-in-One Web Scraping API with AI Extraction

Platform or service · API

Zyte API is a full-stack web scraping API that combines anti-bot bypassing, browser rendering, and AI-powered data extraction in a single endpoint.

Scraping APIs

Exa AI

Semantic Web Search API Powered by AI

Platform or service · API

Exa AI is a modern search engine and web search API that uses semantic embeddings to deliver high-quality, real-time web data for AI applications.

Web Search APIs

Tavily

Real-Time Search API for AI Agents and RAG

Platform or service · API

Tavily is a real-time search engine for AI agents and RAG workflows, delivering fast web search and content extraction through a single secure API.

Web Search APIs

Brave Search API

Independent Web Search API with 35B+ Pages

Platform or service · API

Brave Search API provides enterprise-grade access to an independent index of 35+ billion web pages with real-time results optimized for AI and RAG pipelines.

Web Search APIs

Cloudflare R2

S3-compatible storage with zero egress fees

Infrastructure · API / SaaS

Cloudflare R2 is S3-compatible object storage with zero egress fees, making it ideal for storing and serving frequently accessed scraped data.

Object Storage

AWS Athena

Serverless SQL Queries on Amazon S3

Platform or service · API

Amazon Athena is a serverless, interactive query service that lets you analyze petabyte-scale data in S3 using standard SQL with pay-per-query pricing.

Data Processing

AWS Glue

Serverless Data Integration Service by AWS

Platform or service · API

AWS Glue is a fully managed, serverless ETL service that discovers, prepares, and integrates data from over 100 sources into lakes and warehouses.

ETL Tools

Webshare

Fast and affordable proxy servers

Platform or service · API

Webshare provides self-serve proxy servers across many locations, offering datacenter and residential proxy options for scraping workflows.

Proxy Services

IPRoyal

Premium proxies at affordable prices

Platform or service · API

IPRoyal offers residential, datacenter, ISP, and mobile proxy products with self-service access for scraping workflows.

Proxy Services

Decodo

AI-ready proxy and scraping solutions

Platform or service · API

Smartproxy (now Decodo) offers residential and datacenter proxy products with rotating and sticky sessions for scalable scraping.

Proxy Services

Render

Cloud platform for web services and cron jobs

Infrastructure · API / SaaS

Render is a cloud platform that deploys web services, background workers, and cron jobs with simple scaling for scraping workloads.

Cloud Compute

Railway

Deploy apps from GitHub through configured exporters or custom item pipelines

Infrastructure · API / SaaS

Railway deploys web apps and services from GitHub with auto-scaling, built-in databases, and instant previews for scraping workloads.

Cloud Compute

AWS Lambda

Serverless compute for event-driven pipelines

Infrastructure · API / SaaS

AWS Lambda runs code without managing servers, ideal for event-driven scraping pipelines triggered by schedules, queues, or API calls.

Cloud Compute

Cloudflare Workers

Run serverless code at the edge worldwide

Infrastructure · API / SaaS

Cloudflare Workers lets you run JavaScript and TypeScript at the edge with ultra-low latency for lightweight scrapers, proxies, and APIs.

Cloud Compute

ScraperAPI

Scale data collection with a simple API

Platform or service · API

ScraperAPI provides managed web scraping with proxy rotation, JavaScript rendering and CAPTCHA handling for supported public websites.

Scraping APIs

ScrapingBee

Web scraping API with proxy and browser handling

Platform or service · API

ScrapingBee is a web scraping API that handles headless browsers, rotates proxies, and offers AI-powered extraction so you can focus on your data.

Scraping APIs

DataForSEO

Comprehensive API Stack for SEO and Marketing Data

Platform or service · API

DataForSEO provides APIs for search engine results, keyword data, backlinks and other SEO and digital marketing datasets.

SERP Extraction APIs

CapSolver

AI-powered automatic CAPTCHA solving service

Platform or service · API

CapSolver is an AI-powered CAPTCHA solver that automatically bypasses reCAPTCHA, Cloudflare, AWS WAF, and other challenges for automation.

CAPTCHA Solvers

Oxylabs

Proxy services and web scraping APIs

Platform or service · API

Oxylabs provides proxy services, web scraping APIs and managed browser tools for collecting public web data.

Proxy Services

+1 more

SerpAPI

Real-Time Google Search Results API

Platform or service · API

SerpAPI is a real-time API that delivers structured Google search results, handling proxies, captchas, and rich data parsing automatically.

SERP Extraction APIs

Apify

Full-stack web scraping and data extraction platform

Platform or service · SaaS / API

Apify is a cloud platform for web scraping, browser automation, and data extraction with an Actor marketplace and developer SDKs.

Scraping APIs

BeautifulSoup

Python library for parsing HTML and XML

Library or tool · Library / Self-hosted

Beautiful Soup is a Python library for pulling data out of HTML and XML files, providing Pythonic idioms for navigating and searching parse trees.

HTML Parsers

Modal

Serverless Python platform for AI and data teams

Infrastructure · API / SaaS

Modal is a serverless Python platform that runs scraping, AI, and data workloads at scale with configurable containers and autoscaling.

Cloud Compute

2Captcha

CAPTCHA solving service with API integration

Platform or service · API

2Captcha is a CAPTCHA solving service that bypasses reCAPTCHA, Cloudflare Turnstile, hCaptcha, and other challenges via a simple API.

CAPTCHA Solvers

Zendriver

Async-first undetected browser automation

Library or tool · Self-hosted

Zendriver is a fast, async-first undetected browser automation framework based on nodriver with Docker support for scalable scraping.

Browser Automation

+1 more

Scrapy Cluster

Distributed on-demand scraping with Scrapy

Library or tool · Self-hosted

Scrapy Cluster uses Redis and Kafka to create a distributed, on-demand Scrapy crawling cluster for coordinated large-scale web scraping. Its repository is archived.

Large-Scale & Distributed Crawling

Frontera

Scalable crawl frontier framework

Library or tool · Self-hosted

Frontera is a Python crawl frontier framework for managing when and what to crawl, enabling web crawlers of any scale with Scrapy integration.

Large-Scale & Distributed Crawling