Temporal
Temporal is a durable execution platform that makes distributed applications fault-tolerant by automatically capturing state and recovering from failures.
Temporal is a robust, enterprise-grade durable execution platform that uses recorded workflow history and deterministic replay to recover workflow progress. Activities still need appropriate retries, timeouts, and idempotency.
Key Features:
- Durable Workflows - Workflow history and replay for recovery from failures
- Multi-Language SDKs - Native support for Python, Go, TypeScript, Java, Ruby, C#, and PHP
- Automatic Retries - Built-in timeout and retry logic for failure-prone activities
- Scalable Architecture - Scale workflow execution through configured workers and services
- Full Observability - Inspect workflow history and replay execution for debugging
Whether you're building scraping pipelines, data processing workflows, or distributed systems, Temporal provides fault-tolerant orchestration for supported workloads.
Web scraping workflow
How to use Temporal in a web scraping pipeline
Temporal records workflow progress and executes external work through Activities on workers. It can coordinate a crawl that spans fetching, parsing, waiting and publishing without treating those actions as one uninterrupted process.
A practical workflow
- Start a Workflow with a collection identifier and references to bounded input batches.
- Run HTTP requests, browser sessions and storage operations in Activities; keep Workflow decisions deterministic and pass capture references between stages.
- Publish the dataset through an idempotent Activity and retain its location alongside the workflow's recorded outcome.
When it fits
Use it when collection has several failure-prone stages or must wait for external work before continuing. Separating Activities makes recovery and operational inspection more precise than restarting a large script from the beginning.
Boundaries
Temporal supplies execution coordination, not a browser, scraper or dataset validator. Activity retries can repeat side effects, so stable identifiers and safe destination writes are essential. Keep large page bodies outside workflow history and configure worker capacity, timeouts and target-specific pacing deliberately.
Growing a website? Track its keywords, Google rankings, Search Console insights, and share of voice with bisibility. Free during beta.
Our productRepository
temporalio/temporal
Stars
23.7KForks
2KLast commit
2026-10-15License
MITLanguage
Go
Metrics recorded 2026-10-16
Repository details
License source: Repository license file
Frequently asked questions
What is Temporal orchestration?
Temporal coordinates long-running application workflows and preserves their progress through durable execution. Workflow code schedules Activities, and recorded execution history lets the workflow recover after worker failures. Workflow code must follow Temporal's replay requirements.
Sources (checked ): Temporal Workflows and replay
Is Temporal Server open source?
Yes. The Temporal Server source repository is released under the MIT license. Temporal Cloud is the separately operated managed service; running the open-source server yourself requires managing its infrastructure.
Sources (checked ): Temporal Server MIT license; Temporal deployment options
How do I run Temporal Server locally?
Install the Temporal CLI for your platform and run temporal server start-dev. The development server starts with a default namespace and a Web UI, using an in-memory database by default. The service listens on localhost:7233 and the UI is available at http://localhost:8233. Use the documented persistence option to keep local state between runs.
Sources (checked ): Temporal CLI setup and development server
Can Temporal replace Kafka?
Temporal orchestrates durable application workflows, while Kafka provides event streaming and an event log. Temporal can coordinate work triggered by events, but choosing it does not provide the same event-streaming infrastructure as Kafka. They can be used together for different responsibilities.
Sources (checked ): Temporal Workflows; Kafka event streaming
What are the key differences between Kubernetes and Temporal?
Kubernetes manages containerized workloads and their deployment infrastructure. Temporal manages application workflow execution and recovery through execution history. Kubernetes can run the services and workers used by a Temporal deployment, while Temporal coordinates the application’s workflow logic.
Sources (checked ): Kubernetes overview; Temporal Workflows; Temporal self-hosting
How much does Temporal cost?
Temporal Cloud bills for Actions and active or retained workflow-history storage, with support and add-ons affecting the total. The published entry rate is $50 per million Actions, plus storage and support charges. The MIT-licensed server has no software license fee when self-hosted; infrastructure and operations still cost money. Check the current rate card before estimating a budget.
Sources (checked ): Temporal Cloud rate card; Temporal Server license
Sources
Verified against the temporalio/temporal repository and the go package.