Vector Databases: The Complete Guide for AI Engineers
What vector databases are, why they matter for AI applications, and how to choose the right one.
Vector databases have become the backbone of modern AI applications. Semantic search, RAG systems, recommendation engines, and multimodal search all depend on the ability to efficiently store and retrieve high-dimensional vector embeddings. Understanding vector databases — how they work, what they're good at, and where they fall short — is now a core competency for AI engineers and architects.
What Vector Databases Actually Do
A vector database stores data as high-dimensional numerical arrays (vectors) and enables similarity search — finding vectors that are mathematically 'close' to a query vector. This is fundamentally different from traditional databases, which search by exact match or range queries on structured fields.
The power of vector search is that semantic similarity in the embedding space corresponds to conceptual similarity in the real world. Two documents about the same topic will have similar vectors even if they share no common keywords. This is why vector search dramatically outperforms keyword search for knowledge retrieval, and why it's the foundation of every serious RAG system.
Indexing Algorithms: ANN Search
Searching all vectors exhaustively for the nearest neighbour is O(n) — too slow at scale. Vector databases use Approximate Nearest Neighbour (ANN) algorithms that trade small amounts of recall for massive speed improvements. The dominant approaches are HNSW (Hierarchical Navigable Small World graphs) and IVF (Inverted File Index).
HNSW provides excellent query latency (sub-millisecond for millions of vectors) with high recall and is the default choice for most production deployments. IVF is better for very large datasets (hundreds of millions of vectors) where memory constraints prevent loading all vectors into RAM. Most production vector databases support both.
Choosing a Vector Database
For teams already running PostgreSQL, pgvector is often the right choice — it adds vector search as a Postgres extension, avoiding additional infrastructure. Supabase makes pgvector particularly accessible with managed hosting. For applications where vector search is the primary access pattern and scale is paramount, purpose-built databases offer advantages: Pinecone for managed simplicity, Weaviate for hybrid search (combining vector and keyword), Qdrant for self-hosted control and performance, Milvus for extreme scale.
The most important decision factors are: query latency requirements, dataset size, whether you need hybrid (vector + metadata filter) search, operational complexity tolerance, and whether you need managed hosting or can self-host.
Metadata Filtering and Hybrid Search
Pure vector search returns the most semantically similar results globally. Most production use cases need filtered search — find the most similar documents that also match specific metadata criteria (e.g., 'most similar to this query, but only from documents created this year, in English, for customer segment X').
Metadata filtering is where vector database implementations diverge significantly. Pre-filtering (filter first, then vector search) is fast but can miss relevant results if the filtered set is small. Post-filtering (vector search first, then filter) is more accurate but computationally wasteful. ACORN filtering and segment-based architectures solve this more elegantly — it's worth evaluating specifically for use cases with heavy filtering requirements.