14 · Project: ANN regression
Swap BCE for MSE and remove the sigmoid output — the same architecture now regresses. The pattern generalises across loss/task pairs.
Same ANN, swap BCE for MSE and remove the sigmoid output — now it regresses. The pattern generalizes across loss/task pairs.
Without this:
Without the regression variant, you'd assume neural nets are only for classification.
The previous lesson built a binary classifier. Now we change exactly two things and get a regression model:
- Remove the sigmoid output activation — regression targets are unbounded, so the raw linear output is correct.
- Swap BCE loss for MSE loss — instead of measuring log-likelihood of class labels, we measure mean squared distance from the target values.
Everything else — the hidden layer, ReLU activation, He init, SGD with momentum, the backprop equations — stays exactly the same. This pattern is one of the most powerful lessons in deep learning: the architecture (layer topology, activations, init, optimiser) is almost completely separable from the task (output activation + loss function).
The dataset uses a non-linear target: y = sin(x1) + 0.5*x2^2 + noise. A linear model can't capture this; our ANN can.
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
PyTorch read-along: regression — MSELoss replacing BCELoss, no sigmoid on output
Regression output activation checklist — which activation for which target range
To convert an ANN binary classifier to a regression model, what changes?
- ANN regressor = ANN classifier with sigmoid removed from output and BCE replaced by MSE. Hidden layers are unchanged.
- Output activation selection: none (raw linear) for unconstrained targets; sigmoid for [0,1]; tanh for [-1,1]; relu/softplus for non-negative.
- Linear models fail on non-linear targets (sin, x^2) — the hidden layer's non-linearity is the difference.
- Same pattern for multi-output regression: widen the final nn.Linear layer to n_out > 1 and use MSELoss — PyTorch handles it automatically.
Same pattern works for any single-output regression. For multi-output, just widen the final layer; PyTorch handles it automatically.
If you remove it: You'd assume neural nets are classification-only and miss the unified view that makes transfer learning and multi-task architectures intuitive.