Determinant and Trace

The determinant $\det(A)$ of a square matrix is a single scalar that captures the signed volume scaling factor of the linear map $\mathbf{x} \mapsto A\mathbf{x}$. If $\det(A) = 3$, the map triples areas (in 2D) or volumes (in 3D). If $\det(A) < 0$, it reverses orientation (like a reflection). If $\det(A) = 0$, the map collapses space — volumes become zero because the image is lower-dimensional.

For a $2\times 2$ matrix, $\det\begin{pmatrix} a & b \\ c & d \end{pmatrix} = ad - bc$. In higher dimensions, the determinant expands recursively via cofactor expansion or is computed numerically from the LU decomposition as the product of pivots. Key identities: $\det(AB) = \det(A)\det(B)$, $\det(A^\top) = \det(A)$, and $\det(A^{-1}) = 1/\det(A)$.

The determinant is the gatekeeper of invertibility: $A$ is invertible iff $\det(A) \neq 0$. Singular matrices live on a thin surface in the space of all matrices, the zero set of the determinant polynomial. Numerically, a near-zero determinant warns of ill-conditioning — though the determinant itself can be a misleading size estimate (condition number is a better diagnostic).

The trace $\text{tr}(A)$ is the sum of diagonal entries. Despite its simple definition, it has rich properties: $\text{tr}(AB) = \text{tr}(BA)$ (even when $AB \neq BA$!), it is linear ($\text{tr}(A + B) = \text{tr}(A) + \text{tr}(B)$), and it equals the sum of eigenvalues. Dually, the determinant equals the product of eigenvalues.

Both show up throughout ML. The log-determinant $\log \det(A)$ appears in multivariate Gaussian log-likelihoods and is a key term in normalizing flow training (where the change of variables formula requires $\log|\det(\partial f/\partial \mathbf{x})|$). The trace powers the Frobenius norm $\|A\|_F^2 = \text{tr}(A^\top A)$, used as a matrix-level regularizer. Understanding determinant and trace is the key to reading many probabilistic ML papers.