17 · Prompt vs RAG vs fine-tune
Three ways to adapt an LLM to your problem — and they are NOT interchangeable. Prompting changes behaviour, RAG injects fresh knowledge, fine-tuning bakes in style and skill. A decision framework, not a coin flip.
There are three levers to adapt an LLM. PROMPTING changes behaviour and output format with words alone (no training). RAG injects fresh or private FACTS into the context at query time. FINE-TUNING re-weights the model so a STYLE, FORMAT, or SKILL becomes the default. Each solves a different problem; picking the wrong one wastes money or fails outright.
Without this:
Without the framework people fine-tune to add facts (which doesn't reliably work and goes stale), or stuff a giant prompt full of knowledge that RAG should fetch on demand — paying for the wrong tool on every single call.
You have a base LLM and a problem. Before you write a line of code, you have to choose how you are going to adapt the model — and there are exactly three levers, each fixing a different kind of gap.
1. Prompting — change behaviour with words, no training at all. You edit the system prompt, add few-shot examples, give it a persona or a format spec. This changes how the model behaves on knowledge it already has. It is instant, free to iterate, and reversible. It cannot teach the model facts it never saw.
2. RAG (Retrieval-Augmented Generation) — inject knowledge into the context at query time. You retrieve the relevant chunks from your documents and paste them into the prompt so the model answers from your data. This is the right tool when the gap is missing facts: private docs, fresh data that changes daily, anything that must be cited. The knowledge lives outside the model, so it is always current and you can update it by changing a file.
3. Fine-tuning — actually re-train (a slice of) the weights on your examples, so a style, format, or skill becomes the model's default. This is the right tool when the gap is consistent behaviour the prompt can't reliably enforce: a house writing voice, a strict JSON schema across thousands of calls, a specialized classification skill. It is slow and costly to produce, and it goes stale the moment your facts change — which is exactly why you do NOT fine-tune to add knowledge.
The classic mistake is using fine-tuning to memorize facts. Models are bad at recalling specific facts from fine-tuning, and the moment a fact changes you have to retrain. Facts → RAG. Behaviour/format → prompting first, fine-tuning if prompting isn't reliable enough.
Let's turn the framework into code. The function below takes the requirements of a feature — does it need fresh data? private data? a rigidly consistent format? how much training data do you have? how tight is the latency budget? — and applies the decision rules above to recommend a strategy with reasoning. The point isn't the exact wording; it's that the choice is a deterministic consequence of a few yes/no questions you can answer before building anything.
Python (in browser)
The decision framework as code: facts → RAG, format/behaviour → prompting (cheap) or fine-tuning (when you have data). Note scenario 1 stacks nothing but RAG, while the others lean on prompting/fine-tuning.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
One more pragmatic ordering rule, because it saves the most money: always exhaust the cheaper levers first. Iterating on a prompt takes seconds and costs cents; building a RAG index takes a day; producing a clean fine-tuning dataset and training takes weeks and recurs every time the behaviour needs to change. Climb the ladder only when the rung below genuinely can't reach.
Python (in browser)
The adaptation ladder. RAG is the only lever that adds facts without going stale; fine-tuning is the only one that reliably reshapes format — and the most expensive to maintain.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
A legal team wants the assistant to answer questions about THEIR contracts, which are private and updated weekly. Which lever is the right primary choice?
- Three levers, three gaps: prompting changes behaviour, RAG injects facts, fine-tuning bakes in style/format/skill.
- Facts → RAG (never goes stale, update by editing files); behaviour/format → prompting first, fine-tune only if prompting isn't reliable enough.
- Exhaust the cheaper levers first — prompting iterates in seconds, RAG in a day, fine-tuning recurs in weeks. The levers stack.
Every LLM product makes this call at design time; mature systems stack a fine-tuned format model, a RAG knowledge layer, and a guardrail prompt — each chosen for the gap it closes.
If you remove it: Without the framework teams fine-tune for facts (stale, unreliable) or stuff knowledge into prompts (expensive on every call), wasting weeks and dollars on the wrong lever.