BM25S: Orders of magnitude faster lexical search via eager sparse scoring
2024-07-04Code Available5· sign in to hype
Xing Han Lù
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/xhluca/bm25sOfficialIn paperpytorch★ 1,594
- github.com/conda-forge/bm25s-feedstocknone★ 1
- github.com/xhluca/bm25-benchmarksnone★ 0
Abstract
We introduce BM25S, an efficient Python-based implementation of BM25 that only depends on Numpy and Scipy. BM25S achieves up to a 500x speedup compared to the most popular Python-based framework by eagerly computing BM25 scores during indexing and storing them into sparse matrices. It also achieves considerable speedups compared to highly optimized Java-based implementations, which are used by popular commercial products. Finally, BM25S reproduces the exact implementation of five BM25 variants based on Kamphuis et al. (2020) by extending eager scoring to non-sparse variants using a novel score shifting method. The code can be found at https://github.com/xhluca/bm25s
Tasks
Benchmark Results
| Dataset | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| HotpotQA | BM25S | Queries per second | 20.88 | — | Unverified |
| HotpotQA | Rank-BM25 | Queries per second | 0.04 | — | Unverified |
| HotpotQA | Elasticsearch | Queries per second | 7.11 | — | Unverified |
| Natural Questions | Rank-BM25 | Queries per second | 0.1 | — | Unverified |
| Natural Questions | BM25S | Queries per second | 41.85 | — | Unverified |
| Natural Questions | Elasticsearch | Queries per second | 12.16 | — | Unverified |
| Quora Question Pairs | Elasticsearch | Queries per second | 21.8 | — | Unverified |
| Quora Question Pairs | BM25S | Queries per second | 183.53 | — | Unverified |
| Quora Question Pairs | Rank-BM25 | Queries per second | 1.18 | — | Unverified |
| Quora Question Pairs | BM25-PT | Queries per second | 6.49 | — | Unverified |