Affine Spaces

An affine space is, roughly, a vector space that has forgotten where its origin is. Formally, an affine subspace of a vector space $V$ is a set of the form $\mathbf{x}_0 + U = \{\mathbf{x}_0 + \mathbf{u} : \mathbf{u} \in U\}$, where $\mathbf{x}_0$ is any point and $U$ is a linear subspace. When $\mathbf{x}_0 = \mathbf{0}$, the affine subspace coincides with $U$ itself; otherwise, it is a translate of $U$ that need not pass through the origin.

Familiar examples: a line in $\mathbb{R}^2$ not through the origin is a 1-dimensional affine subspace; a plane in $\mathbb{R}^3$ that is shifted off the origin is a 2-dimensional affine subspace. In general, the solution set of a consistent (non-homogeneous) linear system $A\mathbf{x} = \mathbf{b}$ is an affine subspace: it is a particular solution $\mathbf{x}_p$ plus the null space of $A$.

Affine maps are the natural transformations between affine spaces. They have the form $T(\mathbf{x}) = A\mathbf{x} + \mathbf{t}$, combining a linear map $A$ with a translation $\mathbf{t}$. Unlike pure linear maps, affine maps do not need to fix the origin — they can translate the whole space. Affine maps preserve lines, parallelism, and ratios of distances along a line, but not angles or absolute distances.

A trick that shows up throughout computer graphics and ML: we can embed an affine map into a linear map in one higher dimension. Append a 1 to every vector ($\mathbf{x} \mapsto (\mathbf{x}, 1)$) and use a block matrix $\begin{pmatrix} A & \mathbf{t} \\ \mathbf{0}^\top & 1 \end{pmatrix}$ to capture the entire affine map as ordinary matrix-vector multiplication. This is the same homogeneous coordinate trick used in OpenGL transformation pipelines.

Affine ideas thread through ML more than you might expect. Classical linear regression with a bias term is an affine model: $\hat{y} = \mathbf{w}^\top \mathbf{x} + b$ is linear in $\mathbf{w}$ and $b$, but not linear in $\mathbf{x}$ alone (because of the $+b$). Most neural-network layers are affine-followed-by-non-linear: $\mathbf{h} = \sigma(W\mathbf{x} + \mathbf{b})$. Support vector machines carve out decision boundaries that are affine hyperplanes — solution sets of $\mathbf{w}^\top \mathbf{x} + b = 0$. Understanding affine subspaces is thus the right language for half of modern modeling.