Cloudflare R2
Cloudflare R2 is S3-compatible object storage with zero egress fees, making it ideal for storing and serving frequently accessed scraped data.
Cloudflare R2 is a distributed object storage solution with S3 compatibility and zero egress fees.
Key Features:
- Zero Egress Fees - Download and serve scraped data without paying bandwidth charges.
- S3 Compatibility - Use supported S3 API operations; some operations are unsupported and client configuration may need changes.
- Global Distribution - Store data on Cloudflare's network for low-latency access worldwide.
- Workers Integration - Process and transform stored data at the edge with Cloudflare Workers.
- Usage-Based Costs - Standard storage, requests, and Infrequent Access retrieval have separate charges; check current pricing.
Whether you're archiving scraped datasets, serving media files, or building multicloud storage architectures, Cloudflare R2 provides affordable, fast object storage without egress penalties.
Web scraping workflow
How to use Cloudflare R2 in a web scraping pipeline
Cloudflare R2 stores unstructured objects such as downloaded documents and dataset exports. It can act as a raw-capture archive that separates web acquisition from later parsing and analysis.
A practical workflow
- A separate crawler captures HTML, PDFs or screenshots and records each artifact's source URL and collection time.
- Your application writes the objects to a private R2 bucket under stable, versioned keys and saves a manifest linking keys to the crawl run.
- Parsing or analytical jobs read those captures and publish derived records without refetching the original website.
When it fits
Use it when preserving original inputs matters for debugging, reprocessing or explaining a changed record. A manifest with content hashes makes the relationship between a parsed value and its captured source explicit.
Boundaries
Object storage does not discover URLs, extract fields or validate content. Configure access, retention and the API path used by your application deliberately. Avoid publicly exposing captured documents by default, and keep operational metadata in a queryable store rather than expecting bucket listings to serve as a workflow database.
Frequently asked questions
What are the uses of Cloudflare R2?
R2 stores files as objects in buckets for application data, images, archives and other object-storage workloads. Applications access it through supported APIs. It supplies storage; the application determines upload, access and retention behavior.
Sources (checked ): R2 documentation
Is Cloudflare R2 compatible with S3 storage?
R2 exposes an S3-compatible API, with a published list of supported and unimplemented operations. S3 clients can use supported operations with an R2 endpoint, but compatibility does not cover every Amazon S3 feature.
Sources (checked ): R2 S3 compatibility
Is Cloudflare R2 free to use?
R2 includes a limited monthly Free allowance for Standard storage and operations. Usage beyond the allowance and Infrequent Access retrieval can incur charges. R2 does not charge egress bandwidth, but other Cloudflare services used with it can have separate costs.
Sources (checked ): R2 pricing; Workers pricing
Similar to Cloudflare R2
Sources
Verified against the official Cloudflare R2 website and documentation.