ETL Tools
A curated collection of the best connector and integration tools for extracting and loading scraped datasets between systems, with transformation where supported.
Quick answer
Load scraped records from files, APIs or databases into a destination through connectors and integration pipelines. Airbyte, AWS Glue, dlt and Meltano belong here. Scheduling an ETL job is Workflow Orchestration; transforming data already in a warehouse is Data Transformation & Validation. Amazon Athena queries stored data and is listed directly under Data Processing.
Frequently asked questions
What is ETL and what tools are used for it?
ETL means extract, transform and load: retrieve data, change its shape or quality, then load it into a destination. Tools have different roles. dlt extracts, normalizes and loads data; dbt focuses on transformation in an existing data platform. Choose tools for the stages your pipeline needs.
Sources (checked ): AWS ETL overview; dlt introduction; dbt introduction
Are there any free ETL tools?
Yes. The open-source dlt Python library is licensed under Apache 2.0 and supports extraction, normalization and loading. The library has no software license fee; compute, storage, destination services and paid source APIs can still cost money.
Sources (checked ): dlt introduction; dlt Apache 2.0 license
Is ETL the same as API?
No. ETL is a data pipeline process; an API is an interface through which software exchanges requests and data. An ETL pipeline can extract data from an API, then transform it and load it elsewhere. dlt documents this pattern for REST API sources.
Sources (checked ): AWS ETL overview; dlt introduction
