15 · Vector databases
An in-memory list of embeddings works for a demo. A vector database adds the four things production needs: persistence, metadata filtering, scale, and a clean upsert/query API.
A vector database is the storage layer for RAG: it persists {vector, text, metadata} records, lets you pre-filter by metadata before the similarity search, and scales the nearest-neighbour query that your in-memory loop did by hand — all behind an upsert/query API.
Without this:
Without a vector DB your retrieval lives in a Python list that dies when the process restarts, re-embeds everything on every boot, can't say 'only search this user's documents', and grinds to a halt past a few thousand vectors.
In the RAG lessons you embedded a handful of documents into vectors and searched them with a cosine-similarity loop over a Python list. That is the right mental model — and it is exactly what a vector database does — but a list in memory is missing four things every real app needs:
-
Persistence. A list disappears when the process restarts. Re-embedding your whole corpus on every boot is slow and costs money. A vector DB stores the vectors on disk so you embed once.
-
Metadata filtering. Real records aren't just a vector and a string. They carry metadata:
{"category": "billing", "user_id": 42, "lang": "es", "date": "2026-01"}. You constantly need to search within a subset — only this user's docs, only the FAQ category, only documents newer than a date. A vector DB lets you pre-filter on metadata and then run similarity only over what survives. -
Scale. A brute-force cosine loop is O(N). At a few thousand vectors that's fine; at millions it's far too slow. Vector DBs use approximate nearest-neighbour indexes (next lesson) to answer in sub-linear time.
-
A clean API. Two verbs cover almost everything: upsert (insert-or-update records by id) and query (give a query vector + optional filter, get the top-k back). Every vector DB — Chroma, Pinecone, pgvector, Weaviate, Qdrant — exposes this same shape.
The conceptual leap this lesson nails down is #2, metadata filtering, because it changes the results, not just the speed. Let's build a tiny vector store that stores {vector, text, metadata} and supports a pre-filter.
The store below is a class with the two canonical verbs. upsert keeps records in a dict keyed by id (so re-inserting the same id updates instead of duplicating — that's the up in upsert). query takes a query vector and an optional where filter; it first narrows the candidate set by metadata, then ranks only the survivors by cosine similarity. Watch how the same query returns different documents once we filter by category — the filter changes the answer, not just the runtime.
Python (in browser)
The same query vector returns different documents once a metadata pre-filter is applied — filtering changes the result set, not just the speed.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Notice three properties your earlier in-memory list didn't have, all visible in that one class:
- Upsert by id — calling
upsert("d3", ...)again replaces d3 instead of appending a second copy. That's how you update a document when its source text changes, without rebuilding the index. - Records carry metadata — every vector travels with a dict you can filter on.
- A query contract —
query(vector, top_k, where)is the same signature whether the data lives in this dict, on a Chroma disk file, or in a Pinecone cluster of millions of vectors.
Here is the same flow against a real Chroma client. Read how add/upsert and query(where=...) mirror our class one-to-one — the only difference is that Chroma persists to disk and uses an ANN index instead of a Python loop.
Real vector DBs expose the same two verbs — upsert(ids, embeddings, metadatas) and query(vector, top_k, where). Persistence and ANN are the parts you get for free.
Your support bot must answer using only the documents that belong to the logged-in customer. Which vector-DB feature makes this correct and efficient?
- A vector DB is the storage layer for RAG: it persists {vector, text, metadata} records so you embed once instead of on every boot.
- Metadata pre-filtering changes the result set (search only this user / category / lang), not just the speed — pre-filter, then rank by similarity.
- Every vector DB exposes the same two verbs: upsert (insert-or-update by id) and query (vector + top_k + where filter).
Every production RAG system stores embeddings in Chroma, Pinecone, pgvector, Weaviate, or Qdrant and uses metadata filters for multi-tenant isolation, freshness, and category routing.
If you remove it: Keep retrieval in an in-memory list and you re-embed on every restart, leak data across tenants, and stall once the corpus grows past a few thousand vectors.