Weaviate: Hybrid Search, Vector Indexes, Filters, Reranking, and RAG Architecture
- 3 minutes ago
- 7 min read
Weaviate is frequently summarized as "an HNSW vector database with hybrid search," a description that was accurate for its early releases but understates what the engine actually offers by 2026. The index layer is no longer a single graph structure: Weaviate now ships four distinct index types selected by dataset size and tenancy shape, a filtering path that changed materially with the introduction of ACORN, and a reranking layer that behaves differently depending on whether it runs on Weaviate Cloud or a self-hosted cluster. Treating all of this as one undifferentiated "vector search with keywords" product leads to real misconfiguration — provisioning HNSW's memory footprint for thousands of near-empty tenants, or assuming a reranker module available in self-hosted Docker will also run on the managed cloud offering.
WEAVIATE'S INDEX LAYER: FOUR STRUCTURES CHOSEN BY DATASET SIZE AND TENANCY SHAPE, NOT ONE DEFAULT GRAPH.
How HNSW, Flat, Dynamic, and the newer HFresh index trade memory, latency, and rebuild cost against each other.
HNSW remains the default: a multi-layered graph connecting similar vectors, giving logarithmic-time search and full CRUD support, and it is the right choice for large collections that need high throughput and low latency. Flat is the opposite extreme — a single unsorted list scanned linearly, with a very low memory footprint but search time that grows directly with object count, which makes it suitable only for small or isolated datasets such as individual tenants in a multi-tenant SaaS deployment. Dynamic, an experimental index type introduced in v1.25 that requires async indexing to be enabled, sits between the two: a collection starts as Flat and automatically upgrades to HNSW once its object count crosses a configurable threshold (10,000 objects by default). The upgrade is one-way — a collection does not revert to Flat after deletions bring it back below the threshold — which matters for multi-tenant systems where per-tenant size varies unpredictably and provisioning full HNSW for every tenant regardless of size would waste memory.
HFresh, added in v1.36, is the newest structure and targets memory-constrained deployments at large scale. It keeps only a centroid index — itself built as an HNSW graph over cluster centroids and compressed with rotational quantization — resident in memory, while the actual posting lists are stored on disk compressed to 1-bit rotational quantization. It supports only cosine and squared-L2 distance metrics, and rebalances itself in the background rather than requiring full index rebuilds as data changes. Separately from index type, Weaviate's vector cache has a default ceiling of one trillion objects per collection; during bulk import, vectorCacheMaxObjects needs to be set high enough to hold the full import in memory, because once the cache fills it drops entirely and subsequent queries fall back to disk lookups rather than evicting incrementally.
........
Index type | Memory footprint | Update behavior | Best fit |
HNSW | High — full graph resident in memory | Full CRUD, live graph updates | Large collections needing high QPS and low latency |
Flat | Very low — single linear list | Simple append; scan cost grows with size | Small or isolated collections, single small tenants |
Dynamic (v1.25+, experimental) | Starts as Flat, grows once upgraded | One-way upgrade to HNSW at a configurable threshold (default 10,000 objects); no downgrade after deletions | Multi-tenant setups with unpredictable per-tenant growth |
HFresh (v1.36+) | Low — only centroids in memory; posting lists on disk at 1-bit quantization | Self-balancing in the background; no full rebuilds | Very large datasets under memory constraints; cosine/L2 only |
........
··········
HYBRID SEARCH AND FILTER-AWARE RETRIEVAL: HOW WEAVIATE BLENDS BM25 WITH VECTORS AND KEEPS FILTERS FROM BREAKING THE GRAPH.
Fusion algorithms, the alpha weighting, and the ACORN mechanism that replaced the older pre-filter/post-filter trade-off.
Hybrid search runs a BM25 keyword search and a vector search in parallel and combines the two result sets with a fusion algorithm. RankedFusion, the default through v1.23, scores each object purely by rank position using 1/(rank + 60), which means two objects at the same rank score identically regardless of how different their underlying similarity scores actually were. RelativeScoreFusion, the default from v1.24 onward, instead normalizes each search's raw scores to a 0–1 range before combining them, which retains more of the original signal than pure rank-based fusion. A separate alpha parameter, from 0 to 1, sets the weighting between the two searches — 0 is keyword-only, 1 is vector-only, 0.5 weights them equally — and the server-side default is 0.75, favoring vector search; because not every client library leaves alpha unset, it is worth setting explicitly rather than assuming a library's default matches Weaviate's.
Filtering combined with vector search has historically forced a choice between pre-filtering, which applies the filter first and then searches only the filtered subset — accurate, but scaling linearly as the filtered set grows — and post-filtering, which searches the full HNSW graph first and filters afterward, where the difficulty is guessing how many raw results to over-fetch since filter selectivity depends on data distribution and correlation with the query. Weaviate's older approach blended both, switching between them via a flatSearchCutOff threshold and applying an inline "sweeping" filter during graph traversal; sweeping risks disconnecting the graph and hurting recall when a large share of candidate nodes fail the filter. ACORN, opt-in since v1.27, addresses this without requiring a separate index: it preserves graph connectivity through two-hop expansion around nodes that fail the filter and seeds additional filtered entry points for low-correlation cases, falling back to normal one-hop traversal when the first-hop nodes already pass the filter. Weaviate's own benchmarks report roughly double the throughput at equivalent recall versus sweeping at around 20% filter selectivity, and an order-of-magnitude improvement when filter correlation with the query is very low — these are vendor-reported figures rather than independently verified ones — while sweeping remains faster when filtered and unfiltered results correlate closely, which is why ACORN reverts automatically for high-selectivity filters instead of always running the more expensive path.
··········
RERANKING AND WHAT ACTUALLY RUNS ON WEAVIATE CLOUD VERSUS SELF-HOSTED.
Which reranker integrations are API calls versus containers you must operate yourself — and what the managed service costs at each tier.
Reranking in Weaviate is a second-stage step layered onto vector, BM25, or hybrid search rather than a replacement for first-stage retrieval, since scoring every stored object against a cross-encoder directly would be too slow to run as the primary search; the reranker module retrieves an initial candidate list and then reorders it using a dedicated relevance model. Weaviate documents reranker integrations for Cohere, VoyageAI, Jina AI, NVIDIA, and a local Transformers module. The Transformers integration runs open cross-encoder models — for example cross-encoder/ms-marco-MiniLM-L-6-v2, which Weaviate's own documentation notes scores nearly identically to the larger L-12 model on MS-MARCO (39.01 versus 39.02) — inside a self-hosted container, and is explicitly not available on Weaviate Cloud because it requires spinning up that container directly. The Cohere, VoyageAI, Jina, and NVIDIA integrations are API calls to each provider's hosted reranking model and carry no such restriction, which makes hosting mode rather than model quality the first filter when choosing a reranker for a cloud deployment.
Weaviate Cloud's current pricing spans three tiers. The Free tier is a permanent $0/month plan capped at 100,000 objects, 1 GB memory, and 10 GB disk across a single collection with up to three tenants, plus 2,000 embedding requests per day and 1,000 Query Agent requests per month. Flex starts at $45/month on a pay-as-you-go basis on a shared cluster, with unlimited objects, the full core database toolkit including replication, 30,000 Query Agent requests per month, usage-based embeddings, and a 99.5% uptime SLA. Premium starts at $400/month on a prepaid contract, on shared or dedicated deployment across AWS, GCP, or Azure, with unlimited objects and Query Agent requests, usage-based embeddings, and 99.9–99.95% uptime SLAs depending on deployment mode. Beyond the tier minimums, usage is billed across three additional dimensions: vector dimensions stored (from roughly $0.0047 to $0.0027 per million, varying by tier, region, and compression), storage (roughly $0.12 to $0.15 per GiB), and backups (roughly $0.004 to $0.026 per GiB depending on retention period, from 7 to 45 days) — data transfer is currently free, but Weaviate describes this as a promotional term rather than a permanent one.
........
Plan | Price | Included | Notes |
Free | $0/month, permanent | 100,000 objects, 1 GB memory, 10 GB disk, 1 collection (up to 3 tenants), 2,000 embedding requests/day, 1,000 Query Agent requests/month | Basic email support |
Flex | From $45/month, pay-as-you-go | Unlimited objects, shared cluster, full core toolkit with replication, 30,000 Query Agent requests/month, usage-based embeddings | 99.5% uptime SLA, next-business-day Sev-1 response |
Premium | From $400/month, prepaid contract | Unlimited objects, shared or dedicated deployment across AWS/GCP/Azure, unlimited Query Agent requests, usage-based embeddings | 99.9–99.95% uptime SLA, 4-hour Sev-1 response, dedicated technical account team |
........
··········
WHEN WEAVIATE'S ARCHITECTURE FITS A RAG WORKLOAD — AND WHEN IT DOES NOT.
Matching index choice, filter strategy, and reranker deployment mode to the actual constraint — memory, tenant count, or filter correlation.
The decision that matters most is rarely "HNSW versus something else" in the abstract; it is which of Weaviate's four index types matches the actual shape of the data. A system with many small, unevenly sized tenants should default to Dynamic so that small tenants stay on cheap Flat storage and only graduate to HNSW past the object threshold, rather than paying HNSW's memory cost uniformly regardless of tenant size. A single large collection under sustained, high query volume should stay on HNSW, with quantization (BQ or SQ for a lighter footprint, RQ where the index type supports it, including HFresh's mandatory 1-bit mode) reserved for genuine memory pressure rather than applied by default. When filters correlate poorly with the query vector, enabling ACORN rather than relying on the legacy sweeping path is what keeps recall from collapsing under load — but this is a version-gated, opt-in setting, not automatic behavior on every deployment. Reranker choice should follow hosting mode rather than accuracy claims: a team running on Weaviate Cloud is restricted to the API-based rerankers — Cohere, VoyageAI, Jina, or NVIDIA — since the Transformers cross-encoder module is not offered there at all, regardless of whether a self-hosted benchmark favored it.
··········
FOLLOW US FOR MORE.
·····
DATA STUDIOS
·····
