Scraping Frameworks

All-in-one scraping API with AI extraction. 100% success rate on Google SERP. Formerly Scrapinghub.

The Apify SDK for Python is the official library for creating Apify Actors in Python. It provides useful features like actor lifecycle management, local storage emulation, and actor event handling.

Build production-ready AI agents in Python and TypeScript. Linux Foundation project.

A Python library for automating interaction with websites.

TypeScript library for structured data extraction

Adaptive scraping that learns from website changes

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Python scraper based on AI

Elegant Scraper and Crawler Framework for Golang

🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN

Scrapy, a fast high-level web crawling & scraping framework for Python.