Colly

Colly is a fast, elegant scraping framework for Go that provides a clean interface for building any kind of crawler or spider.

Key Features:

• Speed — Leverages Go's concurrency model for blazing fast parallel scraping
• Clean API — Chainable, callback-based interface for intuitive crawler development
• Cookie Handling — Manages cookies and sessions across requests seamlessly
• Rate Limiting — Built-in request delays and parallelism controls to respect target sites
• Distributed Crawling — Supports distributed scraping with Redis backend
• Caching — Built-in response caching to reduce redundant requests during development

Whether you're a Go developer building data pipelines, an engineer scaling web crawlers, or a researcher collecting structured data, Colly provides a performant and idiomatic solution.