Web Scraping

Rod is a Go library for web automation and scraping that drives browsers via the DevTools Protocol with a simple, chainable API.

Scrapling is an adaptive Python scraping framework that learns from website changes, handling everything from single requests to full-scale crawls.

WebdriverIO is an all-in-one Node.js automation framework for web and mobile testing with smart selectors, component testing, and extensive plugins.

TestCafe is a free, open-source end-to-end testing framework that works across browsers without WebDriver, with intuitive syntax and built-in waiting.

Chromedp is a fast, dependency-free Go package for driving browsers via the Chrome DevTools Protocol for web scraping, testing, and automation.

GoQuery brings jQuery-like syntax to Go for parsing and manipulating HTML documents, making web scraping in Go intuitive and efficient.

Skyvern automates browser-based workflows using LLMs and computer vision, replacing brittle scripts with AI that adapts to any webpage automatically.

Mastra is an all-in-one TypeScript framework for building AI agents and workflows with RAG, memory, MCP support, evals, and an interactive playground.

ScrapeGraphAI extracts structured data from any website using AI, handling proxies, rendering, and adapting to site changes automatically.