16 · Convolution: kernels, strides, and feature maps
2D convolution slides a kernel over an input computing dot products — implement it by hand, apply classical kernels, and master the output-shape formula.
2D convolution slides a kernel (small filter) over an input, computing the dot product at each position — the output is a feature map.
Without this:
Convolution is the central operation; without understanding it, the rest of CNN architecture is opaque.
The word "convolution" sounds intimidating, but the operation is straightforward: take a small matrix of numbers (the kernel or filter), place it on top of a region of the input, multiply each pair of overlapping numbers, and sum them all up. That sum is one output value. Then slide the kernel one position to the right and repeat. Keep sliding until you've covered the whole input — the collection of output values is the feature map.
Formally, for a 2D input I and kernel K, the output at position (i, j) is:
O[i, j] = sum over (m, n): I[i+m, j+n] × K[m, n]
The stride controls how far the kernel moves at each step. Stride 1 means move 1 pixel at a time (maximum overlap, maximum output resolution). Stride 2 means move 2 pixels at a time, which roughly halves the spatial dimensions of the output — a way to downsample while extracting features.
Output shape formula (no padding):
out_size = (in_size - kernel_size) / stride + 1
For example: 28×28 input, 3×3 kernel, stride 1 → (28-3)/1+1 = 26×26 output. Same input, stride 2 → (28-3)/2+1 = 13×13 output.
A small technical note: what DL practitioners call "convolution" is technically cross-correlation — the true mathematical convolution flips the kernel 180° before sliding. Because CNN kernels are learned, the flipping is absorbed into the weights and the distinction is irrelevant in practice.
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Convolution output shape formula and worked examples
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
PyTorch read-along: nn.Conv2d parameters and shapes
A 32×32 input with a 5×5 kernel, stride 1, no padding — what is the output shape?
- 2D convolution slides a kernel over the input, computing the dot product at each position; the result is a feature map highlighting whatever pattern the kernel detects.
- Output shape (no padding): out = (in - k) / stride + 1. Stride 1 → max resolution; stride 2 → ~halved resolution with simultaneous downsampling.
- DL 'convolution' is technically cross-correlation (no kernel flip); the distinction is irrelevant because kernels are learned.
- Integer division in the shape formula means odd remainders silently drop 1 pixel — always use padding to keep shapes predictable.
Conv2d is the most-used layer in computer vision. The same operation, generalized to higher dimensions, runs on 3D medical scans, 1D time series, and even audio waveforms.
If you remove it: Every CNN architecture paper describes shapes in terms of convolution output formulas — AlexNet, VGG, ResNet. Not knowing the formula makes reading any architecture diagram impossible.