1 · The perceptron: deep learning's atomic unit
A single artificial neuron, its weighted sum + activation, and why stacking millions of them becomes a neural network.
A perceptron is a weighted sum of inputs followed by an activation — the same building block stacked millions of times becomes a neural network.
Without this:
Without grasping the single neuron you cannot reason about what a 100-layer transformer is doing.
Welcome to Deep Learning. This track starts where every textbook should: a single artificial neuron.
The roadmap for the track is: perceptron → MLP (multi-layer perceptron) → CNN (convolutional nets) → RNN/LSTM (recurrent nets) → Transformer. Each step adds one idea — but every step is built from the same atom you'll study right now.
A perceptron takes a vector of inputs x, multiplies each by a learned weight w, adds a bias b, and passes the result through an activation function:
output = activation(dot(x, w) + b)
The activation is what gives a neural network its power: without it, stacking layers is equivalent to a single linear transformation. With it, you can approximate any continuous function (Universal Approximation Theorem).
In 1957, Frank Rosenblatt built the first perceptron in hardware. By 1969, Minsky and Papert proved it couldn't learn XOR. By 1986, Rumelhart, Hinton and Williams showed that backprop could train multi-layer nets — and the field was reborn.
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Perceptron history — from 1957 to the 2012 deep learning revolution
Why couldn't a single-layer perceptron learn XOR?
- A perceptron computes `activation(dot(x, w) + b)` — the atomic operation every neural network repeats at scale.
- A single perceptron is a linear classifier; it cannot learn XOR (not linearly separable). Adding a hidden layer creates non-linear decision boundaries.
- The step activation is pedagogically useful but non-differentiable — production nets use ReLU, sigmoid, or GELU so that gradients can flow back during training.
- Minsky & Papert's 1969 XOR limitation applied only to single-layer nets; the 1986 backprop paper unlocked multi-layer learning and ended the AI winter.
Every layer of a deep net is `linear(x) → activation`; the perceptron IS this. Understanding the atom makes every architecture legible.
If you remove it: Deep learning architectures stay opaque — you can call `.fit()` but you can't reason about what the layers are doing or debug them when they fail.