1 · What is an LLM app?
Strip away the magic: a large language model is a function that predicts the next token. Everything an LLM app does is built on top of that one primitive.
A large language model is, mechanically, a next-token predictor: given a sequence of tokens it returns a probability distribution over the next token. An 'LLM app' is the engineering wrapped around that call — context, retrieval, tools, parsing, and guardrails.
Without this:
Without the next-token mental model, LLM behaviour looks like magic and you can't reason about why it repeats, hallucinates, or runs out of context. With it, every technique in this track becomes obvious engineering.
You have already seen, in the Deep Learning track, how a transformer is built. This track is about the layer above the model: the engineering that turns a raw next-token predictor into a useful product — a chatbot, a search assistant, a coding agent.
Strip away every buzzword and a large language model (LLM) is one function:
predict( tokens so far ) → probability for every possible next token
That is the whole interface. The model reads a sequence of tokens (its context), and returns a score for each token in its vocabulary saying how likely that token is to come next. The app picks one token, appends it, and calls the model again. Repeat until a stop signal. Text comes out one token at a time.
Everything you will build in this track is engineering around that single call:
- Context — what tokens you feed in (the system prompt, the conversation, retrieved documents).
- Decoding — how you turn the probability distribution into an actual token (greedy, temperature, top-p).
- Retrieval (RAG) — fetching relevant text to put into the context so the model answers from your data.
- Tools / agents — letting the model decide to call a function, then feeding the result back as more context.
- Evaluation & guardrails — checking the output before you trust it.
The model itself is a frozen black box you call over an API. Your job — the LLM engineer's job — is everything around it.
To make "next-token predictor" concrete, here is a tiny language model you can run. It is a bigram model: it learns, from a small corpus, the probability of each character given the previous one — then generates new text by sampling that distribution one character at a time. A real LLM does the same thing with billions of parameters over subword tokens and a context of thousands of tokens, but the loop is identical.
Python (in browser)
A bigram character model: the same predict→sample→append loop a real LLM runs, shrunk to something you can read end to end.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
In production you don't train the model — you call one. The shape of that call is the same across providers: you send a list of messages (a system instruction plus the conversation) and you get back the model's next message. Read the OpenAI-style call below; we will use this exact shape (a messages list) throughout the track.
Every provider's chat API takes a messages list and returns the next message. Building that list well is the whole game.
Mechanically, what does a large language model compute on each step?
- An LLM is a function: tokens-so-far → probability distribution over the next token. Generation is predict→sample→append, repeated.
- An 'LLM app' is the engineering around that call: context, decoding, retrieval, tools, evaluation, guardrails.
- Every provider's chat API takes a messages list and returns the next message — building that list well is the core skill.
Every chatbot, copilot, and AI search feature is a next-token model wrapped in context-building and output-handling code — the subject of this entire track.
If you remove it: Treat the model as an oracle instead of a next-token predictor and you'll be mystified by hallucinations, repetition, and context limits instead of engineering around them.