19 · Complete CNN architecture: Conv → BN → ReLU → Pool
Assemble the blocks into a complete image classifier: alternating Conv–BN–ReLU–Pool stages, growing channels, shrinking spatial dims, ending in a linear classifier head.
A modern CNN is alternating Conv → BN → ReLU → Pool blocks, gradually reducing spatial size while growing channels, ending in a classifier head.
Without this:
Without the architecture mental model, code like AlexNet/VGG/ResNet looks arbitrary.
You now have every building block: convolution, ReLU, batch normalisation, max pooling. A complete CNN architecture is these blocks arranged in a deliberate pattern:
- Conv → BN → ReLU (feature extraction block) — the workhorse repeated many times.
- MaxPool (or strided conv) — periodic spatial downsampling, typically 2× at a time.
- As spatial dims shrink, channel count grows — more abstract features per location.
- Global Average Pooling — collapses the spatial dims to 1×1, producing a fixed-size feature vector.
- Linear layer — maps the feature vector to class logits.
The intuition for growing channels while shrinking space: early layers detect simple, local patterns (edges, colours). Later layers detect complex, global patterns (textures, object parts, whole objects). A single feature map position in a late layer "sees" a large region of the original image, so it needs more channels to encode the richer content it's summarising.
This pattern was discovered empirically. The milestones:
- LeNet (1989/1998) — first practical CNN on handwritten digits.
- AlexNet (2012) — first GPU-trained deep CNN; won ImageNet by a huge margin; introduced ReLU, dropout, data augmentation at scale.
- VGG (2014) — showed that depth (16-19 layers) with uniform 3×3 kernels beats larger kernels.
- GoogLeNet/Inception (2014) — multi-scale convolutions in parallel ("inception module").
- ResNet (2015) — skip connections let gradients flow across 100+ layers; unlocked modern depth.
- EfficientNet (2019) — compound scaling of depth, width, and resolution together.
Classic CNN architecture diagram: Conv→BN→ReLU→Pool blocks, growing channels, shrinking spatial dims
Parameter count walkthrough — the CNN has ~95k params vs ~820k for an equivalent MLP
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
PyTorch read-along: full SimpleCNN class — Conv/BN/ReLU/Pool/GAP/Linear
Evolution of CNN architectures: LeNet → AlexNet → VGG → Inception → ResNet → EfficientNet
Why do later CNN layers typically have MORE channels but SMALLER spatial dims?
- Standard CNN block: Conv(same padding) → BatchNorm → ReLU → (MaxPool every N blocks). Channels grow (3→32→64→128→...); spatial dims shrink (224→112→56→...).
- CNN architecture evolution: LeNet (1998) → AlexNet (2012, ReLU+GPU) → VGG (2014, 3×3 everywhere) → ResNet (2015, skip connections) → EfficientNet (2019, compound scaling).
- Global Average Pooling → fixed-size vector enables transfer learning: pretrained backbone + new Linear head for any classification task.
- Always print x.shape after each CNN block while building — shape bugs are silent and common, especially at the conv→linear boundary.
ResNet, EfficientNet, MobileNet — every CNN you'll use as a backbone follows this 'Conv block → downsample → grow channels' pattern. Knowing this lets you read any paper's architecture diagram at a glance.
If you remove it: You'd look at a ResNet definition (3 lines of blocks) and have no idea which blocks extract features, which downsample, and why there are 4 stages with doubling channel counts. Every backbone would look like an arbitrary sequence of numbers.