Performance

Method

The results on this page are produced by running elastiknn on the ann-benchmarks benchmark on an AWS EC2 r6i.4xlarge instance. We use task dockerStartBenchmarkingCluster to start a single-node benchmarking cluster task annbRunLocalFashionMnist to run the benchmark.

See the ann-benchmarks directory in the Elastiknn repository for more details.

Results

Fashion MNIST

Model Parameters Recall Queries per Second
eknn-l2lsh L=100 k=4 w=1024 candidates=500 probes=0 0.378 370.991
eknn-l2lsh L=100 k=4 w=1024 candidates=1000 probes=0 0.447 309.856
eknn-l2lsh L=100 k=4 w=1024 candidates=500 probes=3 0.634 293.030
eknn-l2lsh L=100 k=4 w=1024 candidates=1000 probes=3 0.717 247.470
eknn-l2lsh L=100 k=4 w=2048 candidates=500 probes=0 0.767 321.846
eknn-l2lsh L=100 k=4 w=2048 candidates=1000 probes=0 0.847 271.771
eknn-l2lsh L=100 k=4 w=2048 candidates=500 probes=3 0.921 216.381
eknn-l2lsh L=100 k=4 w=2048 candidates=1000 probes=3 0.960 196.737

Comparison to Other Methods

If you need high-throughput nearest neighbor search for batch jobs, there are many faster methods. When comparing Elastiknn performance to these methods, consider the following:

  1. Elastiknn executes entirely in the Elasticsearch JVM and is implemented with existing Elasticsearch and Lucene primitives. Many other methods use C and C++, which are generally faster than the JVM for pure number crunching tasks.
  2. Elastiknn issues an HTTP request for every query, since a KNN query is just a standard Elasticsearch query. Most other methods operate without network I/O.
  3. Elastiknn stores vectors on disk and uses zero caching beyond the caching that Lucene already implements. Most other methods operate entirely in memory.
  4. Elastiknn scales horizontally out-of-the-box by adding shards to an Elasticsearch index. Query latency typically scales inversely with the number of shards, i.e., queries on an index with two shards will be 2x faster than an index with one shard. Few other methods are this simple to parallelize.