18 · Channels: RGB images and multi-filter convolutions
Real images have 3 channels (RGB) — each CNN filter is 3D (h × w × in_channels) and stacking many filters builds the depth dimension of feature maps.
Real images have 3 channels (RGB) — each conv filter is actually 3D (h × w × in_channels), and you have many such filters.
Without this:
Grayscale-only mental model breaks when you hit ImageNet or any real photo dataset.
Every lesson so far treated images as 2D grids of numbers. Real images are 3D tensors: height × width × channels. For standard colour photos that means 3 channels — Red, Green, Blue. The pixel at position (y, x) is not a single number but a triplet (R, G, B) representing the intensity of each colour component.
This has a direct consequence for CNN filters. In the grayscale case a 3×3 filter has 9 weights. For an RGB input, the same "3×3" filter is actually 3×3×3 = 27 weights — a small cube that spans the full depth of the input. The filter computes one dot product between all 27 of its weights and the corresponding 3×3×3 patch of the input, producing a single output value. The output feature map from one such filter is still 2D.
To produce multiple feature maps — detecting different features at each spatial location — you use multiple filters. If you use 32 filters each of size 3×3×3, you get 32 separate 2D feature maps, which are then stacked into a 3D output tensor of shape (out_h, out_w, 32). The 32 output channels are the "depth" of the feature map, completely analogous to the 3 channels of the input RGB image.
So the general shape of a conv layer is:
- Input: (H, W, in_channels)
- Filter bank: (out_channels, kH, kW, in_channels)
- Output: (out_H, out_W, out_channels)
Where each output channel is the response of one learned filter.
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
PyTorch read-along: nn.Conv2d with RGB input — weight shapes and forward pass
A Conv2d with in_channels=64, out_channels=128, kernel_size=3 has how many weight parameters?
- RGB image = (H, W, 3) tensor; a CNN filter is a 3D cube (kH, kW, C_in) that produces one 2D feature map per filter via a single dot product across all channels.
- Filter bank shape: (C_out, kH, kW, C_in) — C_out filters each spanning C_in channels; output shape is (out_H, out_W, C_out).
- Parameter count: C_out × C_in × kH × kW weights + C_out biases. For (3→32, k=3): 864+32 = 896 total.
- NumPy/PIL uses (H, W, C); PyTorch uses (C, H, W) — always permute with `tensor.permute(2, 0, 1)` when loading custom image data.
Every image classifier handles RGB. ImageNet pretrained models (ResNet, VGG, EfficientNet) all expect (3, 224, 224) inputs normalized to roughly zero-mean unit-variance.
If you remove it: The moment you look at any ImageNet model's first layer shape — (64, 3, 7, 7) in ResNet-50 — it's opaque without understanding that 3 is C_in and 64 is C_out. Every pretrained backbone shape becomes unreadable.