9 · Latent factors
Stop comparing users to users. Instead, give every user and every item a short vector of hidden 'taste dimensions' — the predicted rating is just their dot product.
Represent each user and each item as a k-dimensional latent vector. The predicted rating is their dot product: R ≈ P Qᵀ. The factors are 'taste dimensions' (action-vs-romance, indie-vs-blockbuster) discovered automatically — and they make the huge rating matrix approximately LOW-RANK.
Without this:
Memory-based CF (Ch.3) stores the whole matrix and recomputes similarities every time — it doesn't scale and chokes on sparsity. Latent factors compress millions of users/items into tiny vectors that generalize across the gaps.
Chapter 3's collaborative filtering was memory-based: keep the whole matrix, find lookalike users (or items) on the fly, average their ratings. It works, but it doesn't scale and it dies on sparsity — two users who share no rated items have undefined similarity.
Matrix factorization takes a completely different, model-based view. The key insight: a rating isn't random — it's driven by a few hidden dimensions of taste. Maybe one axis is action ↔ romance, another is blockbuster ↔ indie. We don't name these axes; the model discovers them. The idea is to give:
- every user
ua length-kvectorP[u]— how much they care about each latent dimension, and - every item
ia length-kvectorQ[i]— how much that item expresses each dimension.
Then the predicted rating is simply their dot product:
R[u,i] ≈ P[u] · Q[i] = Σ_f P[u,f] · Q[i,f]
A user who loves action (high on the action factor) paired with an item that is strongly action (high on the same factor) yields a big dot product → a high predicted rating. In matrix form the whole prediction is R ≈ P Qᵀ, where P is users × k and Q is items × k.
Because k is tiny (think 10–200) compared to the number of users or items, this says the rating matrix is approximately low-rank: its essential information lives in a handful of dimensions. That is the entire premise of the next two lessons — how to find good P and Q.
Below we hand-craft factors so you can read them, reconstruct R = P Qᵀ, and watch a user's vector explain which items they'll like.
Python (in browser)
Hand-crafted action/romance latent factors. The predicted rating is the dot product of a user and item vector — and the whole matrix is low-rank (rank ≤ k).
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
In a latent-factor model, how is the predicted rating R[u,i] computed?
- Each user and item becomes a k-dimensional latent vector; the predicted rating is their dot product: R ≈ P Qᵀ.
- Latent factors are 'taste dimensions' (action↔romance, blockbuster↔indie) discovered automatically — never hand-labeled in practice.
- Because k is tiny, the rating matrix is approximately LOW-RANK — its signal lives in a handful of dimensions.
The same dot-product-of-embeddings idea powers Netflix's MF models, two-tower retrieval at YouTube, and word2vec — embeddings are latent factors everywhere in ML.
If you remove it: Without latent factors you're stuck recomputing pairwise similarities over a giant sparse matrix — slow, and unable to generalize across users who never overlap.