Manticore Search
Manticore Search is an open-source search database for full-text search and analytical queries through SQL and HTTP APIs.
Manticore Search is a fast, open-source search database built for high-performance full-text search and analytics.
Key Features:
- SQL Interface - Query indexed data with familiar SQL syntax alongside a REST API.
- Real-Time Indexing - Ingest and search scraped data immediately without batch delays.
- Columnar Storage - Store large datasets efficiently with built-in columnar storage for analytics.
- Auto-Sharding - Distribute indexes across nodes automatically for horizontal scaling.
Whether you're indexing scraped web content, building log analytics, or powering search over large datasets, Manticore Search provides the speed and simplicity your search workloads demand.
Web scraping workflow
How to use Manticore Search in a web scraping pipeline
Manticore Search supports real-time tables whose documents can be added, updated and deleted. It can provide a mutable search layer for a corpus that a separate scraper refreshes over time.
A practical workflow
- A crawler extracts text and attributes using stable document identifiers derived from your entity model.
- An indexing job writes accepted documents into a real-time table and applies updates or removals when the canonical dataset changes.
- A search application queries that table and shows matches with their source URL and collection context.
When it fits
Use this when search should reflect a continuing stream of collected-document changes instead of only periodic full exports. Keep the identifier policy consistent so a recrawl updates the intended document rather than creating an accidental duplicate.
Boundaries
The real-time table manages indexed data, not website acquisition or extraction correctness. You must operate the service, design its fields and manage memory and disk behavior. Keep raw captures and authoritative records elsewhere, and explicitly propagate source removals instead of assuming missing crawl results should delete indexed documents.
Growing a website? Track its keywords, Google rankings, Search Console insights, and share of voice with bisibility. Free during beta.
Our productRepository
manticoresoftware/manticoresearch
Stars
12.1KForks
647Last commit
2026-10-15License
GPL-3.0Language
C++
Metrics recorded 2026-10-16
Repository details
License source: Repository license file
Sources
Verified against the manticoresoftware/manticoresearch repository and the npm package.