FAISS: Approximate Nearest Neighbor Search, IVF, HNSW, and Product Quantization

FAISS (Facebook AI Similarity Search) is an open-source C++ library, with complete Python and C wrappers, for exact and approximate nearest-neighbor search over dense vectors. Meta AI Research released it in February 2017 under the MIT license; the original authors credited are Hervé Jégou, Matthijs Douze, and Jeff Johnson, and Meta continues to maintain it. FAISS is a library, not a database: it exposes no network protocol, no query language, no authentication, and no metadata planner. An application links it in-process, on CPU or GPU, and builds whatever persistence, filtering, and serving logic the deployment needs around it. In retrieval-augmented generation pipelines this typically means an application layer owns document text and metadata in a separate store, calling FAISS only for the nearest-neighbor step over embedding vectors and passing back internal integer IDs to resolve against that store.

········

FAISS INDEX TYPES: FLAT, IVF, HNSW, AND PRODUCT QUANTIZATION.

How each index structure trades memory footprint and build cost against query latency and recall.

IndexFlatL2 and IndexFlatIP perform exhaustive brute-force search: exact results, no training step, and cost that scales linearly with collection size. FAISS's own indexing guide reports roughly 9.1 seconds per query batch against SIFT1M (1 million 128-dimensional vectors) with a flat index, which is why flat indexes are used mainly as a correctness baseline or for collections in the low tens of thousands of vectors.

IndexIVFFlat partitions the vector space into clusters with a k-means-trained coarse quantizer (for example IVF4096,Flat creates 4,096 clusters), then restricts each query to the nprobe nearest clusters instead of the whole collection. Raising nprobe trades query latency for recall; the coarse quantizer itself must be trained on a representative sample before vectors can be added. On the same SIFT1M reference, an IVFFlat index holds the dataset in roughly 520 MB plus about 8 MB of index overhead.

IndexHNSW builds a multi-layer proximity graph and is controlled by M (graph connectivity), efConstruction (build-time search width), and efSearch (query-time search width). It is the fastest of FAISS's CPU index types at a given recall level — FAISS's benchmark notes roughly 0.02 ms per query for HNSW versus roughly 0.14 ms for IVFFlat at comparable recall on SIFT1M — but graph structures consume more memory per vector than IVF lists and, unlike IVF, cannot be trivially rebalanced after large deletions.

Product quantization (PQ) compresses each vector into a short code by splitting it into subvectors and quantizing each independently against a learned codebook; IndexPQ applies this alone, while IndexIVFPQ combines coarse IVF partitioning with PQ compression of the residuals for both speed and memory reduction at large scale. OPQ (optimized product quantization) applies a learned rotation before PQ to reduce quantization error at the same code length. On SIFT1M, FAISS's guide gives OPQ32_128,IVF4096,PQ32 a footprint near 40 MB — 32 bytes of PQ code plus an 8-byte ID per vector — against roughly 520 MB for uncompressed IVFFlat, at a measurable recall cost that narrows as code length increases. More recent fast-scan 4-bit PQ variants in FAISS improve on classic PQ and scalar-quantizer accuracy at comparable speed across most operating points, per the same guide.

........

Index type	Search behavior	Approx. memory (SIFT1M reference)	Primary recall/speed control	Typical fit
IndexFlatL2 / IndexFlatIP	Exact, linear scan	~512 MB (raw floats)	None — always exact	Correctness baseline; small collections
IndexIVFFlat	Approximate, cluster-pruned	~520 MB + ~8 MB overhead	nprobe	Balanced recall/speed on mid-size datasets
IndexHNSW	Approximate, graph traversal	Higher than IVF per vector	efSearch, efConstruction, M	Lowest latency at a given recall, read-heavy workloads
IndexIVFPQ / OPQ+IVFPQ	Approximate, cluster-pruned + compressed	~40 MB (OPQ32_128,IVF4096,PQ32)	nprobe, code length (M subquantizers)	Billion-scale collections under memory pressure

........

········

GPU ACCELERATION AND SCALING TO BILLION-VECTOR DATASETS.

What changes when indexes move off CPU, and where the NVIDIA cuVS integration applies.

FAISS ships four GPU index types — GpuIndexFlat, GpuIndexIVFFlat, GpuIndexIVFScalarQuantizer, and GpuIndexIVFPQ — designed as drop-in replacements for their CPU counterparts. A StandardGpuResources object manages scratch memory, reserving 512 MiB by default on GPUs with 4 GB or less, 1,024 MiB up to 8 GB, and up to 1,536 MiB above that; this is adjustable, including down to zero. GPU indexes carry specific constraints: both k (neighbors returned) and nprobe are capped at 2,048, and GpuIndexIVFPQ code sizes are restricted to a defined set of values between 1 and 96 bytes, with sizes of 56 bytes or more forced into float16 mode because of shared-memory limits. Per FAISS's GPU documentation, single-GPU search runs roughly 5–10x faster than the equivalent CPU index, the workload is generally memory-bandwidth bound rather than compute bound, and throughput benefits require batching queries rather than issuing them one at a time.

Multi-GPU deployments follow one of two strategies. Replication copies the full index onto each GPU and parallelizes queries across them, yielding near-linear speedup — FAISS's documentation cites 6–7x with 8 GPUs — but does not increase the dataset size a single node can hold. Sharding splits the dataset across GPUs so aggregate capacity grows, at the cost of sub-linear speedup because each query must consult every shard. Storing vectors and computing distances in float16 instead of float32 roughly halves GPU memory use; FAISS's own testing reports recall as largely unaffected, and GPUs from the Pascal generation onward get additional throughput from native float16 support.

In 2025 Meta and NVIDIA integrated NVIDIA's cuVS library into FAISS starting with version 1.10, distributed as an optional conda package that lets a deployment choose between FAISS's native GPU kernels and cuVS-accelerated ones for the same index types. The integration accelerates IVF Flat and IVF PQ, and introduces CAGRA (CUDA ANN Graph) as a GPU-native graph index positioned as an alternative to CPU-based HNSW. According to benchmarks Meta and NVIDIA published on their engineering blog — run with the cuVS-bench tool on H100 GPUs against Intel Xeon Platinum CPUs, across datasets from 5 million to 100 million vectors at 95% recall@10 — build times improved by up to 2.7x for IVF Flat, up to 4.7x for IVF PQ, and up to 12.3x for CAGRA versus HNSW, while search latency improved by up to 1.9x for IVF Flat, up to 8.1x for IVF PQ, and up to 4.7x for CAGRA versus HNSW. These figures come from the vendors' own benchmark methodology rather than independent third-party testing, and results vary by dataset and recall target. Separately, Meta has stated FAISS has indexed up to 1.5 trillion 144-dimensional vectors in internal production use — a vendor-reported scale claim about Meta's own infrastructure, not a published, reproducible benchmark.

········

FAISS AGAINST MANAGED AND EMBEDDED VECTOR STORES.

Where a raw algorithm library sits relative to systems that also handle storage, filtering, and network access.

FAISS's IVF, HNSW, and PQ implementations are widely used as reference implementations, and several vector databases have historically embedded FAISS itself as one of their index backends rather than reimplementing the algorithms. That reuse means the algorithmic ceiling — recall at a given latency for a given index type — is often similar across tools built on comparable libraries. What differs is the operational layer FAISS deliberately omits: durable storage with crash recovery, concurrent-write safety, a metadata query planner, multi-tenant access control, a network interface, and horizontal scaling across nodes. FAISS's only filtering primitive is IDSelector, a mechanism for restricting search to (or excluding) a set of internal integer IDs computed by the caller before or during the search call; it is not a query language and does not index metadata fields itself, so combining vector similarity with structured filters (date ranges, category, tenant ID) requires the calling application to compute the eligible ID set from an external store, or accept the recall cost of over-fetching and post-filtering.

........

System	Deployment model	Metadata filtering	Persistence	Network access	Cost model
FAISS	Embedded library, in-process	IDSelector only (caller-computed ID sets)	Manual (write_index / read_index to a file)	None built in	MIT license, free; infrastructure and engineering cost only
Milvus	Standalone or distributed service	Native, indexed scalar fields	Built in (metadata store + object storage)	gRPC / REST	Apache 2.0; self-hosted or managed tiers
Supabase pgvector	PostgreSQL extension	Full SQL WHERE clauses, joins	PostgreSQL durability (WAL, backups)	PostgreSQL wire protocol	PostgreSQL license; Supabase hosting fees or self-hosted
Chroma	Embedded (in-process) or client-server	Native metadata filters	Built in (local persistence or server)	Optional HTTP server	Apache 2.0; free self-hosted, managed option available

........

········

WHEN TO USE FAISS DIRECTLY VERSUS A VECTOR DATABASE.

What FAISS leaves for the caller to build, and the concrete condition for choosing it over a managed store.

Adopting FAISS directly means accepting several unsolved problems as engineering scope rather than configuration. There is no built-in persistence server: index state lives in memory and must be explicitly serialized with write_index and reloaded with read_index, with no automatic backup, point-in-time recovery, or replication — a process crash between writes loses unsaved state. There is no concurrent-write protection: adding vectors from multiple threads or processes to the same index requires external coordination. There is no authentication, multi-tenancy, or row-level access control; isolating tenants means running separate indexes or filtering by caller-supplied ID sets. There is no horizontal scaling coordinator — sharding across machines, routing queries, and merging results is application code. None of this makes FAISS unsuitable for production; it means the operational surface that a database like Milvus, Qdrant, or pgvector provides out of the box has to be built and maintained separately, which is a reasonable trade when the win is control over exact index parameters, GPU placement, and update patterns, or when an existing system already provides storage and only needs a fast nearest-neighbor kernel bolted on.

The decision reduces to a specific condition: use FAISS directly when the vector collection fits within the RAM or GPU memory of machines fully under the deployment's control, when metadata filtering needs can be expressed as an ID allowlist or denylist computed upstream rather than as arbitrary structured queries, and when the team has the capacity to own serialization, backup, and serving code for the lifetime of the system. When concurrent writes, rich metadata filtering, multi-tenant isolation, or cross-node scaling become requirements rather than conveniences, route those through a system that implements them natively — reserving FAISS for the cases where its raw index performance, benchmarked against that system's native index on the actual dataset size and dimensionality in question, is measurably better.

FOLLOW US FOR MORE.

·····

DATA STUDIOS

·····

datastudios.org