Sign In
Favicon of Apache StormCrawler

Apache StormCrawler

PublishedLast updated

Build scalable Java crawlers on Apache Storm with low-latency processing, flexible customization, and an Apache-2.0 license.

Visit Apache StormCrawler
Screenshot of Apache StormCrawler website

Sources

github.com

Verified against the apache/stormcrawler repository.

Apache StormCrawler helps teams build low-latency, scalable web crawlers on Apache Storm. It gives Java developers a customizable open-source foundation for distributed crawling.

Key Features:

  • Apache Storm Base - Build crawler pipelines on top of Apache Storm.
  • Scalable Crawling - Designed for distributed crawling workloads.
  • Low-Latency Processing - Supports crawler use cases that need fast handling.
  • Java Customization - Write crawler logic in Java to fit your workflow.
  • Open Source - Apache-2.0 license, 981 GitHub stars, and 282 forks.
Categories:
Distributed Crawling
Features:
Distributed CrawlingConcurrency ControlReal-Time StreamingOpen SourceJava Customization
Tags:
high-concurrencyjavaopen-source

Share:

  • Stars

    981

  • Forks

    282

  • Last commit

    17 days ago

  • License

    Apache-2.0

  • Language

    Java
View Repository
Scraping · 0/38
1,284 rows

Something new is scraping

We’re building our own product. Get early access.

38 pages · 0 errors · 4.2s
See what’s coming

Similar to Apache StormCrawler

Favicon

 

  
  
Favicon

 

  
  
Favicon