6 · A vector index from scratch
Embeddings plus cosine similarity become a search engine when you store vectors with their text and return the top-k closest. Build a tiny VectorIndex and watch semantic search emerge.
A vector index is just a list of (vector, metadata) records plus a search method: embed the query, score every stored vector by cosine similarity, and return the top-k with their original text. That tiny data structure — store, then rank by proximity — is the retrieval engine inside every RAG system and vector database.
Without this:
Without an index you'd re-embed and re-scan everything from scratch on every query, and you'd have no way to pull back the original text behind a winning vector — so retrieval can't feed an LLM anything readable.
Now we assemble the pieces from lessons 4 and 5 into the core abstraction of this whole track: a vector index. It does two things:
add(id, vector, text)— store a vector alongside its metadata (here, the original text, but in real apps also things like a document URL, author, or timestamp).search(query_vector, k)— score every stored vector against the query by cosine similarity and return the top-k records, highest score first.
That's it. There is no neural network in the index itself — the intelligence lives in the embedding model that produced the vectors. The index is just bookkeeping plus a sort: keep the vectors, and when asked, rank them by proximity.
A crucial detail the index makes obvious: you store the vector for search but return the text for use. The vector is how you find the right record; the text is what you actually feed to the LLM (in RAG) or show to the user (in search). Losing that mapping — vector back to text — would make retrieval useless.
Below, VectorIndex is built on a NumPy matrix of stored vectors so one @ (matrix-vector product) scores the entire corpus at once. We include a tiny embed() helper right in the cell so the whole search pipeline — embed → store → query → rank → return text — runs end to end.
Python (in browser)
A from-scratch VectorIndex: add() stores vector+text, search() ranks the whole corpus with one matrix product and returns the top-k original texts — semantic search in ~25 lines.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Notice what just happened: a query like "python bug in the code" surfaced both d1 and d6 even though neither matches the query word-for-word — they share the vocabulary directions python/code/bug/error, so cosine ranks them on top. Swap our toy embed() for a real embedding model and this exact VectorIndex becomes a genuine semantic search engine. In production you'd reach for a vector database, but its mental model is precisely what you just built.
FAISS in production mirrors our toy index exactly: normalize, add() vectors, search() for top-k by inner product (= cosine), then map ids back to text. Same interface, industrial scale.
In the VectorIndex, why do we store the original text alongside each vector instead of just keeping the vectors?
- A vector index is a list of (vector, metadata) records plus a search method — store, then rank by cosine proximity.
- search(query, k) embeds the query, scores all stored vectors (one matrix product for unit vectors), and returns the top-k with their original text.
- Store the vector to search, return the text to use — the vector→text mapping is what makes retrieval feedable to an LLM. Brute-force scan is fine to ~100k vectors; ANN indexes scale further with the same interface.
This add/search top-k structure IS the retrieval step of RAG and the core of every vector database (Pinecone, Weaviate, pgvector, FAISS, Chroma) — the next chapter wires it straight into an LLM prompt.
If you remove it: With no index you can't retrieve relevant context efficiently, so RAG, semantic search, and recommendation have nothing to rank — the LLM is left to answer from memory alone, where hallucinations live.