Elasticsearch: Hybrid Search, BM25, Dense Vectors, RRF, and Reranking
Updated: Sep 15
Elasticsearch handles three distinct retrieval mechanisms inside a single query layer: lexical scoring against an inverted index (BM25), approximate nearest-neighbor search over dense vector embeddings, and learned sparse retrieval through Elastic's own ELSER model. Since the introduction of the retriever framework, these mechanisms compose through a declarative request structure — a top-level retriever wraps sub-retrievers, Elasticsearch executes BM25 and kNN search in parallel, and a fusion stage merges their rankings with Reciprocal Rank Fusion (RRF) or a weighted linear combination.
A reranking stage can then sit on top of the fused list, either using Elastic's own cross-encoder model or delegating to Cohere, Google Vertex AI, or a custom Hugging Face model through the Inference API. For teams already storing operational, log, or application data in Elasticsearch, this means retrieval, hybrid fusion, and reranking run inside one cluster without standing up a separate vector database — at the cost of managing quantization settings, ML node sizing, and inference billing that a dedicated vector store typically abstracts away.
········
RETRIEVAL MECHANISMS: BM25, DENSE VECTORS, AND LEARNED SPARSE ENCODING.
How each field type is indexed, queried, and sized before any fusion happens.
BM25 is Elasticsearch's default full-text scoring algorithm on text fields: term frequency with saturation and field-length normalization, tunable through the k1 and b parameters, running against a Lucene inverted index. It needs no embedding model, indexes cheaply, and scores exact or near-exact term matches highest — which is also its limit against paraphrased or conceptually related queries.
The dense_vector field type stores embeddings produced by an external or Elastic-hosted model. element_type can be float (4 bytes per dimension), byte (1 byte), bit (1 bit), or bfloat16 (2 bytes). The indexing algorithm is set through index_options.type: hnsw is the default for byte and bit vectors; for float vectors under 384 dimensions, int8_hnsw is the default, giving roughly 4x memory reduction through scalar quantization; at 384 dimensions and above, bbq_hnsw (better binary quantization) becomes the default, cutting memory by roughly 96% (32x) but requiring more than 64 dimensions; int4_hnsw reaches roughly 87% (8x) reduction but needs an even dimension count. Flat variants (flat, int8_flat, int4_flat, bbq_flat) skip the HNSW graph for brute-force search on smaller datasets, and bbq_disk pushes quantized vectors to disk-backed storage but requires an Enterprise subscription. Default HNSW parameters are m=16 and ef_construction=100; dimensions cap at 4096; supported similarity metrics are l2_norm, cosine (default for unmapped float arrays), dot_product, and max_inner_product.
ELSER (Elastic Learned Sparse EncodeR) expands query and document text into weighted term sets stored via the sparse_vector field and queried with a sparse_vector query — an inverted index rather than a graph. Version 1 remains in technical preview; version 2 is generally available and is the recommended version. It's tuned for English text; Elastic recommends the E5 model for non-English semantic search. Running it requires a dedicated ML node (minimum 4GB on Elastic Cloud Hosted without autoscaling) and a subscription tier that includes semantic search — it is not available on every plan.
The semantic_text field type wraps these mechanisms by auto-chunking text and calling a configured inference endpoint at ingest time, removing the need for a manual ingest pipeline. Left unset, it defaults to Elastic's own Elastic Inference Service (EIS) on Elastic Cloud and serverless projects, or to a preconfigured ELSER endpoint (.elser-2-elasticsearch) on self-managed clusters; an explicit inference_id can point it at any other configured model. Elastic recommends pinning inference_id explicitly in production, since relying on the default can cause mapping inconsistencies across version upgrades.
........
Retrieval mechanism | Field / query type | Index structure | Typical use |
Lexical (BM25) | text, match/multi_match | Lucene inverted index, term frequency + length norms | Exact terms, rare-term precision, IDs and codes |
Dense vector (kNN) | dense_vector, knn retriever | HNSW graph or flat; optional int8/int4/bbq quantization | Semantic similarity on model embeddings |
Learned sparse (ELSER) | sparse_vector query | Inverted index over expanded term-weight pairs | Semantic + term overlap, English content |
Managed ingestion (semantic_text) | semantic_text, match/semantic | Auto-chunked; backing store depends on the inference model | Simplified setup, deferred model choice |
........
········
FUSING RANKINGS: THE RETRIEVER FRAMEWORK AND RECIPROCAL RANK FUSION.
How BM25 and kNN results merge into one ranked list before any reranking runs.
Elasticsearch's retriever framework represents a search as a nested JSON tree rather than a flat query. A request specifies either a conventional query or a retriever, never both; a top-level retriever — standard, knn, rrf, linear, or text_similarity_reranker — can wrap one or more child retrievers, letting a single request describe retrieval, fusion, and reranking as one pipeline instead of separate round-trips.
The rrf retriever fuses ranked lists using each document's rank position rather than its raw score. Its rank_constant defaults to 60 and must be at least 1; its rank_window_size defaults to 10 and must be at least as large as the requested result size. When per-retriever weights are set (default 1.0 each), the final score is a weighted sum of each sub-retriever's RRF contribution, letting BM25 or vector results dominate the fused ranking when tuned deliberately. Because RRF works purely on rank position, it discards the magnitude gap between a strong BM25 match and a weak one — a known limitation when one signal is far noisier than the other for a given query.
The linear retriever is Elastic's alternative for cases where that discarded magnitude matters: it normalizes and combines raw scores directly under explicit per-retriever weights instead of ranks, at the cost of needing scores that are actually comparable across retrievers.
The common pipeline nests an rrf retriever around a standard (BM25) sub-retriever and a knn sub-retriever, executes both, fuses the two ranked lists, and returns rank_window_size candidates — which then feed a reranking retriever for final ordering.
········
[[ADPLUS_300x250_1]]
RERANKING THE FUSED LIST: ELASTIC RERANK, COHERE, AND THIRD-PARTY CROSS-ENCODERS.
Where the final ordering comes from once BM25 and vector rankings are already merged.
The text_similarity_reranker retriever wraps a child retriever — typically the output of an rrf stage — and reorders its top rank_window_size documents (default 10) against an inference_text value, scoring each document-query pair with a cross-encoder rather than comparing precomputed embeddings.
Elastic Rerank, the default model behind the preconfigured .rerank-v1-elasticsearch endpoint, is a cross-encoder built on the DeBERTa-v3 architecture with disentangled attention, at 184 million parameters. Elastic states — as a vendor benchmark claim, not independently verified — that it matches the ranking quality of 2-billion-parameter alternatives and improves ranking quality by roughly 40% over raw BM25 across its own evaluation suite. It runs on an ML node (minimum 4GB standalone, 8GB when co-deployed with ELSER or another model), is English-only, and caps input at 512 tokens; it has been available since Elasticsearch 8.17 and on serverless projects.
Because reranking runs one inference pass per candidate at the configured depth, Elastic's own guidance recommends shallow reranking — around the top 30 candidates — when inference runs on CPU, to keep latency and compute cost bounded.
Cohere Rerank and Google Vertex AI's ranking service are reachable through the same text_similarity_reranker retriever by pointing inference_id at a Cohere or Vertex AI inference endpoint configured with the rerank task type — the request shape is unchanged; only the execution location moves from the local ML node to an external API call, which adds network latency and shifts billing to that provider. A custom cross-encoder, such as Hugging Face's cross-encoder/ms-marco-MiniLM-L-6-v2, can also be uploaded through Eland and served as a text_similarity NLP task on an ML node, avoiding both Elastic's model and any external dependency at the cost of operating the deployment directly.
........
Reranker | Model type | Execution location | Constraints / cost note |
Elastic Rerank | Cross-encoder, DeBERTa-v3, 184M params | Local ML node (Inference API) | English only, 512-token input cap; vendor-reported ~40% NDCG gain over BM25 |
Cohere Rerank | Cross-encoder (proprietary) | External API via inference endpoint | Multilingual; billed by Cohere per request, adds network latency |
Google Vertex AI ranking | Cross-encoder (proprietary) | External API via inference endpoint | Billed by Google Cloud, adds network latency |
Custom Hugging Face model (via Eland) | Any HF text-similarity cross-encoder | Local ML node (manual upload) | Model-dependent limits; no external dependency, full ops ownership |
........
········
CHOOSING A CONFIGURATION FOR A RAG PIPELINE.
Where hybrid search and reranking earn their indexing and compute cost, and where they don't.
Cost accumulates at three separate points: storage and indexing for whichever field types are enabled, ML node time for ELSER, semantic_text inference, or Elastic Rerank, and — if routed externally — a separate bill from Cohere or Google Vertex AI. On Elasticsearch Serverless, Elastic's published list rates are approximately $0.14 per VCU-hour for ingest, $0.09 per VCU-hour for search, $0.07 per VCU-hour for machine learning workloads, $0.047 per GB-month for storage, and $0.05 per GB for egress (with 50GB free on vector-enabled profiles); the Elastic Inference Service lists inference at roughly $0.08 per million tokens, separate from the Elastic Managed LLM used for generation-style AI features, billed at $4.50 per million input tokens and $21 per million output tokens. These are Elastic's own "as low as" starting rates and vary by region and commitment level, not fixed prices for a given workload.
Quantization choice interacts directly with that storage bill: bbq_hnsw's roughly 96% memory reduction is the default and generally the right choice for float embeddings at 384 dimensions or above, while lower-dimension embeddings fall back to int8_hnsw's smaller but still substantial 4x reduction — running uncompressed float vectors at scale is rarely justified once one of these quantized modes is available.
Route to a plain BM25 standard retriever when the corpus is small, queries are keyword-precise — IDs, product codes, exact phrases — and no embedding model is already part of the stack. Add an rrf retriever combining a BM25 sub-retriever and a knn sub-retriever once queries carry paraphrase or conceptual intent that term matching misses, keeping rank_window_size shallow (roughly 50 or fewer candidates). Gate a text_similarity_reranker stage behind that fusion only when ranking precision at the top of the list — not recall — is the actual bottleneck: use Elastic Rerank when content is English and keeping inference on-cluster is preferred, and fall back to an external Cohere or Vertex AI endpoint when multilingual coverage or an existing rerank contract already governs the rest of the retrieval stack.
········
FOLLOW US FOR MORE.
·····
DATA STUDIOS
·····
