Search as a service architecture

SeekStorm architecture — what’s different?

Highly integrated approach

To achieve the best scalability and performance while keeping operational costs (and prices) at a minimum, we had to build everything from scratch: database, search server, crawler. Only by getting full control of every byte processed, transferred, and stored, we were able to strictly optimize for search, performance, and efficiency, getting rid of boundaries between different packages and jettisoning unnecessary ballast.

Because SeekStorm utilizes the infrastructure more efficiently, we can pass on the lower operational cost and give you more performance at a lower price.

Novel Log-structured merge-tree database

To achieve low latency under high concurrent load, for billions of documents, in a multi-tenant architecture, we had to rethink search-specific database architecture and take the Log-structured merge-tree (LSM-tree) to the next level.

We built a resilient, reliable and highly parallelized architecture with uniform term-partitioned sharding, n-fold double buffering, zero-copy, lock-free synchronization, thread-per-core architecture optimized for high throughput, short average- and tail-latencies for high-concurrency and web-scale scenarios.

What makes SeekStorm's technology unique?

SeekStorm was designed from scratch with speed AND web-scale scalability in mind.

Until now users had to make a choice between fast, but size-limited (128 GB) and expensive ($10/GB) RAM-based indices - or large (100 TB), cost-efficent ($0,1/GB) SSD-based indexes with limited performance. SeekStorm's novel architecture combines unrivaled scalability with blazing-fast performance without relying on expensive, index size limiting RAM. SeekStorm's bets on its novel database and index architecture and smart algorithms instead of expensive bare metal and brute force approach. This allows comparable performance and better scalability at a much lower cost.

It is not only extremely fast (single-digit milliseconds), but also maintains its speed for unlimited index size (spanning the whole SSD, far beyond RAM size). This applies both for single multi-terabyte indices and multiple large multi-tenancy indices on a single machine.

The combination of instant search for unlimited index size and true real-time search with zero soft-commit delays sets SeekStorm apart from the competition.

This unique technological foundation enables SeekStorms unrivaled scalability and performance/price efficiency without sacrificing profitability and economical sustainability.

High-level architecture diagram


SeekStorm Search as a Service architecture with search API and crawl API