rclone

Rclone is a versatile, feature-rich command-line tool that manages files across 70+ cloud storage providers.

Key Features:

• Universal Compatibility — Sync to S3, Google Cloud, Azure, Dropbox, and dozens more with a single tool.
• Familiar CLI — Unix-style commands like sync, copy, mount, and ls make adoption effortless.
• Encryption — Encrypt scraped data at rest on any cloud provider with built-in crypto.
• Bandwidth Control — Throttle transfers and resume interrupted uploads over unreliable connections.
• Mount as Disk — Mount any cloud storage as a local filesystem on Windows, macOS, or Linux.

Whether you're migrating scraped datasets between providers, archiving crawl results, or mounting remote storage locally, Rclone provides the reliability and flexibility you need.