Colly

Colly is a fast, elegant scraping framework for Go that provides a clean interface for building any kind of crawler or spider.

Key Features:

• Speed - Leverages Go's concurrency model for blazing fast parallel scraping
• Clean API - Chainable, callback-based interface for intuitive crawler development
• Cookie Handling - Manages cookies and sessions across requests seamlessly
• Rate Limiting - Built-in request delays and parallelism controls to respect target sites
• Distributed Crawling - Supports distributed scraping with Redis backend
• Caching - Built-in response caching to reduce redundant requests during development

Whether you're a Go developer building data pipelines, an engineer scaling web crawlers, or a researcher collecting structured data, Colly provides a performant and idiomatic solution.