Milvus: Distributed Vector Search, HNSW, IVF, Quantization, and Scaling
Updated: Sep 9
Milvus is an open-source, Apache 2.0-licensed vector database originally built by Zilliz, hosted under the LF AI & Data Foundation since January 2020 and holding graduate status since June 2021. It separates storage and compute into independently scalable services and ships in three forms: embedded (Milvus Lite, for local prototyping), standalone (single-node), and distributed cluster (multiple specialized node types behind a proxy layer). Zilliz also operates Zilliz Cloud, a managed version of the same engine. The current stable release line is Milvus 3.0, which reached general availability on July 29, 2026 and replaced the 2.6.x series as the default recommendation for new deployments; 2.6 remains widely deployed in production and introduced most of the quantization and storage-tiering mechanics discussed below. This article covers how Milvus indexes vectors (HNSW, IVF, DiskANN, GPU indexes), how the distributed architecture scales ingestion and query independently, what RaBitQ quantization and tiered storage change in memory footprint and cost, and what self-hosting versus Zilliz Cloud costs in practice.
········
INDEX TYPES: HNSW, IVF, AND DISKANN TRADE-OFFS.
How each index structures the search space, and where its assumptions break down under real query patterns.
Milvus builds every index from a combination of a data structure, an optional quantization step, and an optional refinement pass, and it exposes a large catalog of named indexes that are really combinations of those three elements. IVF (inverted file) partitions the vector space into centroid-defined buckets at build time and, at query time, searches only the nprobe buckets nearest the query vector; it is throughput-friendly on large datasets and predictable in memory use, but recall depends heavily on how many buckets get probed, and it degrades on filtered queries with low selectivity. HNSW builds a multi-layer proximity graph and walks it greedily from an entry point, controlled by M (graph connectivity) and ef_search/ef_construction (search and build breadth); it gives strong recall at low top-k and low latency without a training/clustering step, at the cost of holding the full graph plus vectors in memory unless combined with quantization. DiskANN stores a Vamana graph on disk rather than in RAM, trading some latency for the ability to serve datasets far larger than available memory — Milvus's own guidance places it as the preferred choice when data cannot fit in RAM but SSD latency is acceptable. GPU-accelerated variants (GPU_CAGRA, GPU_IVF_FLAT, GPU_IVF_PQ, GPU_BRUTE_FORCE) shift the build and search workload onto CUDA-capable hardware for datasets where CPU search becomes the bottleneck. A hard constraint applies across all of them: Milvus currently allows only one index per field per collection, so the choice is a single structural commitment per vector field, not something that can be A/B tested by stacking multiple indexes side by side.
........
Index | Structure | Best fit | Memory profile (Milvus example, 1M × 128-dim vectors) |
FLAT | Brute-force, no index | Exact recall required, or filter ratio >95% | Full vector storage, no compression |
HNSW | Multi-layer proximity graph | Low-latency search at small top-k, data fits in RAM | ~640 MB (≈128 MB graph + ≈512 MB vectors) |
HNSW_PQ / HNSW_SQ | HNSW graph + quantized vectors | RAM-constrained HNSW deployments | ~136 MB with PQ compression |
IVF_FLAT | Centroid clustering, uncompressed vectors | Baseline throughput/recall reference | Full vector storage per cluster |
IVF_PQ | Centroid clustering + product quantization | Large datasets, memory-constrained | ~11 MB (no refinement) to ~62 MB (10% raw refinement) |
IVF_RABITQ | Centroid clustering + 1-bit binary quantization | Aggressive memory reduction with refinement pass to recover recall | Down to ~3% of FP32 footprint before refinement |
DISKANN | Vamana graph, disk-resident | Billion-scale datasets that don't fit in RAM | Graph on SSD, small in-memory cache |
GPU_CAGRA / GPU_IVF_PQ | GPU-native graph or IVF variants | High-QPS search where CPU is the bottleneck | GPU memory-bound, separate from host RAM |
........
········
DISTRIBUTED ARCHITECTURE: STORAGE-COMPUTE SEPARATION AND SCALING.
Why compute and storage scale independently in cluster mode, and what that buys and costs operationally.
In cluster deployment, Milvus splits responsibilities across a proxy (stateless request routing), coordinator services (collection/schema metadata, and in 3.0 increasingly consolidated), query nodes (serve search and scalar-filter execution against loaded segments), data nodes (handle ingestion, flush, and compaction), and index nodes (build indexes on sealed segments). Object storage (S3, MinIO, GCS, or Azure Blob) holds vector and scalar data durably; etcd holds cluster metadata; and a write-ahead layer absorbs incoming writes before they're flushed to object storage. Through 2.5, that write-ahead layer depended on an external message queue (Kafka or Pulsar); Milvus 2.6 introduced a Streaming Node component and a new WAL implementation called Woodpecker that removes the external message-queue dependency, giving a diskless architecture that Zilliz reports as outperforming Kafka in their own throughput tests (their published figures: roughly 130 MB/s reference throughput for Kafka versus 450 MB/s for Woodpecker in local mode and 750 MB/s in S3 mode — vendor-reported, not independently verified). This split lets query capacity and ingestion capacity scale on separate node pools: a write-heavy ingestion burst does not need to provision additional query nodes, and a read-heavy analytics workload does not need additional data nodes.
Scaling is bounded by concrete, documented ceilings rather than being unlimited: a single collection supports up to 16 shards and up to 1,024 partitions, an instance supports up to 65,536 collections, and each collection is limited to 64 fields and one index per vector field. Zilliz's own 2.6 release material separately claims support for "up to 100K collections in a single cluster" in multi-tenant configurations — a vendor figure that sits above the documented 65,536 collection ceiling and should be read as a cluster-level multi-tenancy claim rather than a per-instance limit override; teams evaluating heavy multi-tenancy should confirm the current number against the release notes for the specific version they deploy. Loaded data is capped at roughly 90% of total query-node memory to preserve headroom for query execution, which means memory planning has to account for index overhead, not just raw vector size. Milvus offers four consistency levels — Strong, Bounded Staleness, Session, and Eventually — selectable per query, trading read-after-write guarantees for lower latency. The operational cost of this architecture is real: a self-managed cluster means running and monitoring etcd, object storage, the WAL/streaming tier, and several distinct node roles, which is meaningfully more moving parts than a single-binary vector store, and is the primary reason most teams below billion-vector scale either use Milvus standalone or move to Zilliz Cloud rather than operating the full cluster topology themselves.
········
QUANTIZATION AND TIERED STORAGE: MEMORY, RECALL, AND COST.
What RaBitQ, scalar/product quantization, and hot/cold tiering change in memory footprint and query throughput.
Scalar quantization (SQ8) stores each vector dimension as an 8-bit integer instead of 32-bit float, cutting memory roughly 75% with a modest, generally predictable recall cost. Product quantization (PQ) splits each vector into subvectors and encodes each against a learned codebook, reaching 4–32x compression at the cost of a larger and less predictable recall hit that depends on codebook quality and data distribution. RaBitQ, added in Milvus 2.6, is a 1-bit binary quantization method that Zilliz reports achieves up to 32x memory compression versus FP32; used alone it is aggressive enough that recall levels off around 76% on Zilliz's internal 1M-vector, 768-dimension benchmark (vendor-reported, not independently reproduced here). Paired with an SQ8 refinement pass — re-scoring a shortlist of candidates against higher-precision vectors — the same benchmark reports 94.7% recall at 864 queries per second, versus 236 QPS for uncompressed IVF_FLAT at comparable recall, a claimed 3x-plus throughput gain. Separately, Zilliz's 2.6 announcement claims "72% memory reduction with RaBitQ ... while delivering 4x faster queries" and, in another passage of the same material, a 97% memory reduction (to roughly 3% of original footprint) before refinement — these are vendor figures from different parts of the same release announcement and are not fully reconciled with each other, so they should be treated as directional rather than as a single consistent number.
Tiered storage, also introduced in 2.6, automatically classifies data segments as hot or cold based on access frequency, keeping hot segments in memory/SSD and moving cold segments to object storage, with Zilliz claiming roughly 50% cost savings from the resulting reduction in provisioned memory (vendor claim). Int8 vector compression for HNSW indexes offers a middle ground — 8-bit storage for graph-indexed vectors without the accuracy volatility of PQ or the extreme compression (and extreme recall sensitivity) of RaBitQ.
........
Method | Compression vs. FP32 | Recall behavior | Typical use |
None (FLAT/uncompressed) | 1x | Reference/exact baseline | Small datasets, exact recall requirements |
SQ8 | ~4x (75% reduction) | Modest, predictable loss | General-purpose memory savings |
PQ | 4x–32x | Variable, codebook-dependent | Large datasets tolerant of recall variance |
RaBitQ (no refinement) | Up to 32x | Recall plateaus near 76% (vendor benchmark) | Extreme memory constraints, recall-tolerant workloads |
RaBitQ + SQ8 refinement | ~28% of original footprint (vendor figure) | 94.7% recall (vendor benchmark), near IVF_FLAT parity | Large-scale deployments needing both memory savings and recall |
Int8 compression (HNSW) | ~4x | Lower variance than PQ | RAM-constrained HNSW deployments |
........
········
DEPLOYMENT, PRICING, AND WHEN MILVUS FITS.
Self-hosted OSS versus Zilliz Cloud costs, the documented operational ceilings, and a concrete choice rule.
Self-hosted Milvus carries no license cost — it is Apache 2.0 — but standalone mode is the realistic entry point for most teams; cluster mode requires operating etcd, object storage, the WAL/streaming tier, and multiple node roles, which implies either existing Kubernetes expertise or acceptance of that operational overhead. Zilliz Cloud, the managed offering, publishes four tiers as of this writing (list pricing, subject to change and confirmed against Zilliz's own pricing page): a Free tier at $0/month with 5 GB storage, 2.5 million vCUs included per month, and up to 5 collections on community support; a Serverless/Standard tier billed pay-as-you-go with a $0 base, positioned for prototypes; a Dedicated Enterprise tier starting from $197/month with a 99.95% uptime SLA, RBAC, SSO, audit logs, and 24/7 support; and a Business Critical tier at custom/contact-sales pricing with a 99.99% multi-replica SLA and HIPAA-eligible configurations for regulated environments. Dedicated clusters are additionally priced by cluster type: performance-optimized (roughly 2M vectors, 500–1,500 QPS) from $63 per million vectors per month; capacity-optimized (roughly 8M vectors, 100–300 QPS) from $16 per million vectors per month; and tiered-storage clusters (roughly 40M vectors, 10–50 QPS) from $5 per million vectors per month — all vendor list prices, not verified independently, and dedicated clusters scale up to 2,048 compute units.
The documented per-instance ceilings worth planning around before committing to Milvus: 32,768 maximum vector dimensions, 65,535-character VARCHAR fields, 64 fields and one index per vector field per collection, 16 shards and 1,024 partitions per collection, 65,536 collections per instance, and a 16,384 cap on both top-k and concurrent query batch size (nq). None of these are exotic for typical RAG or semantic-search workloads, but multi-tenant designs that plan one collection per tenant should model growth against the collection ceiling explicitly rather than assuming it is unbounded. Choose Milvus, self-hosted or via Zilliz Cloud, when the deployment genuinely needs billion-scale vector counts with independently scaled ingestion and query throughput, GPU-accelerated indexing, or multi-tenant collection counts in the tens of thousands; choose a simpler single-binary vector store instead when the working set stays in the low millions of vectors and the team is not prepared to operate etcd, object storage, and a WAL/streaming tier alongside the database itself.
FOLLOW US FOR MORE.
·····
DATA STUDIOS
·····
