35 · Attention Is All You Need
Assemble the full Transformer end to end and see why it beat RNNs: full parallelism across the sequence and a constant-length path between any two positions.
The full Transformer = embeddings + positional encoding → N encoder blocks → N decoder blocks (masked self-attn + cross-attn + FFN) → linear → softmax. It beat RNNs through full parallelism and constant path length.
Without this:
Without seeing the whole stack assembled, the pieces (heads, PE, blocks, masks) stay disconnected and you can't reason about why Transformers scaled where RNNs stalled.
Time to assemble everything from this chapter into the complete machine from "Attention Is All You Need" (Vaswani et al., 2017). End to end, the data flows like this:
- Input embedding + positional encoding (lesson 32). Tokens become vectors; the position signal is added.
- N encoder blocks (lesson 33). Each: multi-head self-attention → Add & Norm → FFN → Add & Norm. The output is a set of encoded representations of the source.
- N decoder blocks (lesson 34). Each: MASKED multi-head self-attention → Add & Norm → CROSS-attention into the encoder output → Add & Norm → FFN → Add & Norm.
- Linear + softmax over the vocabulary. The final decoder representations are projected to vocab-sized logits, and softmax turns them into a probability distribution over the next token.
Why did this beat RNNs so decisively? Two structural wins:
-
Full parallelism across the sequence. An RNN must compute hidden state
h_tfromh_{t-1}— strictly sequential, so a length-nsequence takesnsequential steps you cannot parallelize. Self-attention computes ALL positions at once as a couple of big matrix multiplies. On modern GPU/TPU hardware, that is the difference between crawling and flying. -
Constant path length between any two positions. In an RNN, information from position 1 reaches position 100 only after passing through 99 intermediate states — a path of length
O(distance)along which gradients fade and information is diluted. In self-attention, ANY two positions interact directly in a single attention step: the path length isO(1), constant, no matter how far apart they are. Long-range dependencies become easy to learn.
These two properties — parallelism and constant path length — are why the Transformer, despite being conceptually simple, unlocked the scale that produced BERT, GPT, and everything after.
Full Transformer: source/target embeddings + PE → N encoder blocks → N decoder blocks (masked self-attn + cross-attn + FFN) → Linear → Softmax over vocab
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
What is the Transformer's key training advantage over an RNN?
- The full Transformer is embeddings + positional encoding → N encoder blocks → N decoder blocks (masked self-attn + cross-attn + FFN) → Linear → softmax over the vocabulary.
- It beat RNNs through full parallelism (no sequential h_t ← h_{t-1}) and a constant O(1) path length between any two positions, vs an RNN's O(distance) path.
- Per-layer cost is O(n²·d) for self-attention vs O(n·d²) for recurrence — self-attention wins on ops when n < d — but the O(n²) memory wall spawned FlashAttention, sliding-window, and linear-attention variants.
This exact architecture (or one of its encoder-only / decoder-only variants) is the backbone of essentially every state-of-the-art NLP, speech, and vision model since 2018.
If you remove it: You'd know the individual parts but not why the assembled Transformer scaled where RNNs stalled, nor why context-length and the O(n²) wall dominate so much modern LLM research.