22 · Capstone: a mini RAG-agent
Assemble the whole track into one pipeline: ingest → chunk → embed → index → retrieve → grounded generate → guardrail → eval, end to end on a tiny knowledge base.
A RAG-agent is a pipeline of the pieces from this track: ingest and chunk documents, embed and index them, retrieve by cosine similarity, generate an answer grounded only in the retrieved text, guard the I/O, and score groundedness. Each stage is simple; the value is in wiring them together correctly.
Without this:
Knowing each stage in isolation isn't the same as shipping one. The capstone forces the integration: where chunk size meets retrieval quality, where retrieval meets grounding, where a guard catches what eval would only flag after the fact.
Time to put it all together. A RAG-agent is the canonical LLM-app architecture, and you now have every piece:
- Ingest raw documents (your knowledge base).
- Chunk them into retrieval-sized pieces.
- Embed each chunk into a vector.
- Index the vectors for fast lookup.
- Retrieve the chunks most similar to the query (cosine similarity).
- Generate an answer that uses only the retrieved text (a grounded answerer).
- Guardrail the output (here: a grounding gate — refuse if not supported).
- Eval the result (groundedness score from lesson 20).
The cell below wires all of that end to end over a tiny four-document knowledge base. The embedder is a deterministic bag-of-words vector (real systems use a trained model, but the retrieval mechanics are identical), and the "generator" is a mock grounded answerer: instead of calling an LLM, it stitches together sentences drawn only from the retrieved chunks — a deterministic stand-in that, by construction, can't hallucinate. That lets us see the whole flow run, reproducibly, in the browser.
Python (in browser)
The whole track in one cell: the query retrieves the refund chunk, the mock answerer returns only retrieved text, and the grounding gate passes it (groundedness 1.0).
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Where to go next. You have a working mental model of the full stack — now deepen each stage with real components: swap the bag-of-words embedder for a sentence-transformer model, the linear scan for a vector database (FAISS, pgvector, Pinecone), and the mock answerer for a real grounded LLM call with a citation-forcing prompt. Add the guardrail and eval layers from lessons 20–21 into your CI so every change is measured. Then extend the agent: multi-step tool use, query rewriting, re-ranking, and conversation memory. Every one of those is the same pattern you just built — a deterministic pipeline wrapped around a next-token predictor.
You started the track by stripping an LLM down to "a function that predicts the next token." You're ending it having wrapped that function in retrieval, generation, guardrails, and evaluation — the complete anatomy of a shippable LLM app.
Production swaps the toy parts for real ones (sentence-transformer embeddings, FAISS, an LLM), but the eight-stage shape — and the grounding gate and groundedness eval — stay identical to what you built.
In the capstone, why is the mock answerer described as something that 'by construction can't hallucinate'?
- A RAG-agent is an eight-stage pipeline: ingest → chunk → embed → index → retrieve → grounded generate → guardrail → eval.
- Grounding the generator in retrieved text (and gating on a groundedness threshold) is what keeps a RAG answer trustworthy.
- Production swaps the toy parts (bag-of-words, linear scan, mock answerer) for real ones (trained embeddings, vector DB, an LLM) — the architecture is unchanged.
This is the reference architecture behind documentation assistants, support copilots, and AI search — built with LangChain/LlamaIndex over a vector store, guarded and continuously evaluated.
If you remove it: Skip the integration and you have isolated tricks but no shippable system; the failures live in the seams — chunk size vs retrieval, retrieval vs grounding, grounding vs guardrail.