Windmill
Windmill is an open-source platform for building workflows, internal tools, and data pipelines with full code flexibility and a visual flow builder.
Windmill is a fast, open-source platform that combines the power of code with a visual builder to create workflows, internal tools, and data pipelines. Supporting Python, TypeScript, Go, and SQL, Windmill is built for developers who need real code capabilities without platform overhead.
Key Features:
- Multi-Language Scripts - Write in Python, TypeScript, Go, PHP, Bash, or SQL with full LSP support
- Visual Flow Builder - Orchestrate scripts into high-performance DAG workflows
- Auto-Generated UIs - Instantly create interfaces from script parameters
- Self-Hostable - Deploy with Docker on infrastructure you operate
- Job Engine - Runs scripts and workflow tasks through workers
Whether you're a developer, data engineer, or ops team, Windmill provides the flexibility to automate any workflow at scale.
Web scraping workflow
How to use Windmill in a web scraping pipeline
Windmill runs scripts and assembles them into flows with inputs and outputs. A scraping team can use it as an operator-facing entry point for controlled collection and post-processing jobs.
A practical workflow
- Accept a crawl manifest and output prefix through a script or flow input rather than accepting arbitrary executable code.
- Run your Python or TypeScript scraper, then connect a validation script and a loading script as explicit stages.
- Return a dataset reference and execution result that an operator can inspect before using or publishing the data.
When it fits
This suits internal collection workflows where developers maintain the code but other team members need a repeatable way to run it. Distinct scripts keep acquisition, normalization and destination writes understandable and independently maintainable.
Boundaries
The chosen worker environment must contain the required libraries and browser binaries. Windmill's runtime is not itself a hosted scraping product. Limit accepted URLs and credentials, make repeated writes idempotent and store large captures externally; a successful script run does not certify the extracted data's accuracy.
Growing a website? Track its keywords, Google rankings, Search Console insights, and share of voice with bisibility. Free during beta.
Our productRepository
windmill-labs/windmill
Stars
18.2KForks
1.1KLast commit
2026-10-15License
AGPL-3.0 (default source); Apache-2.0 (listed clients/OpenAPI)Language
Rust
Metrics recorded 2026-10-16
Repository details
License source: Repository license file
Frequently asked questions
What is Windmill used for?
Windmill turns scripts into workflows and internal tools. It generates interfaces from script parameters, lets developers chain scripts into flows and runs jobs on workers. Flows can be triggered through schedules, webhooks and other integrations.
Sources (checked ): Windmill main concepts
How do I install Windmill?
Windmill’s self-hosting documentation provides Docker Compose and Kubernetes deployment options. For the Docker Compose setup, use the project’s documented configuration files to start the application, workers and PostgreSQL, then open the web interface and complete the instance setup.
Sources (checked ): Windmill self-hosting
What is a Windmill server?
In a self-hosted Windmill deployment, API servers handle application requests and workers execute jobs. The frontend provides the interface, while PostgreSQL stores application data and the job queue.
Sources (checked ): Windmill stack
Is Windmill free to use?
Windmill offers a free self-hosted Community Edition with feature and usage limits. Paid cloud and enterprise options are also available. Review the current edition comparison and license terms when choosing a deployment, especially if you plan to redistribute or offer Windmill as a service.
Sources (checked ): Windmill edition comparison; Windmill distribution terms
What are some alternatives to Windmill?
Retool covers overlapping needs for building internal applications, while Prefect focuses on orchestrating Python workflows. These tools overlap with different parts of Windmill, so compare the interface-building and workflow-execution features your project actually needs.
Sources (checked ): Windmill concepts; Retool internal applications; Prefect workflow orchestration
Sources
Verified against the windmill-labs/windmill repository and the npm package.