8 · Similarity metrics & prediction
Cosine vs Pearson vs adjusted cosine — the similarity you pick changes who your neighbors are. Then turn the weighted-sum predictions into a top-N recommendation list.
The neighbor that drives a CF prediction depends entirely on the similarity metric: cosine compares raw rating vectors, Pearson correlation compares ratings after subtracting each user's mean, and adjusted cosine subtracts each item's mean. Different metrics rank the same neighbors differently — and the weighted-sum prediction, sorted over unseen items, becomes your top-N list.
Without this:
Pick the wrong similarity and your neighborhoods are full of users who merely share rating habits, not taste — and the resulting top-N list recommends the wrong items with false confidence.
Memory-based CF has two design choices: which similarity metric, and how to turn predictions into a ranked list. The metric matters more than people expect — it decides who your neighbors are.
Three classic metrics:
- Cosine similarity — the angle between two raw rating vectors. Simple, but a generous user (all 4-5) and a harsh user (all 1-2) can look similar just because both vectors point 'positive'.
- Pearson correlation — cosine after subtracting each user's mean. It measures whether two users deviate from their own averages in the same direction, so it cancels the generous-vs-harsh bias. (User-user CF almost always uses Pearson, i.e. the mean-centering from Lesson 6.)
- Adjusted cosine — for item-item CF: subtract each user's mean from the columns before comparing items, removing the user-bias that plain item cosine ignores.
The unifying rule: subtract a bias (user mean or item mean) before measuring similarity. Pearson removes user bias; adjusted cosine removes user bias from item comparisons.
Once you have a similarity and the weighted-sum prediction pred(u,i) = Σ sim·(rating) / Σ |sim|, recommending is easy: compute pred(u,i) for every item u hasn't rated, sort descending, and take the top N. That sorted slice is the user's recommendation list — the whole point of the system. Below we compute cosine and Pearson user similarity on one matrix, show they can rank neighbors differently, then emit a top-N list.
Python (in browser)
Cosine vs Pearson user similarity on the same matrix: subtracting each user's mean (Pearson) re-ranks neighbors, because cosine is fooled by generous/harsh grading habits.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Now the payoff: turn the weighted-sum predictions into a ranked top-N list for one user. We predict every unrated item using Pearson-weighted neighbors, then sort. This sorted list is literally what the system shows the user.
Python (in browser)
From predictions to a top-N list: predict every unrated item with Pearson-weighted neighbors, sort descending, and the top slice is the user's recommendation list.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
What is the key difference between cosine similarity and Pearson correlation for comparing two users' ratings?
- Cosine compares raw vectors; Pearson = cosine on mean-centered ratings (removes user bias); adjusted cosine removes user bias from item-item comparisons.
- Different similarity metrics can rank the same neighbors differently, so the metric choice directly changes the predictions.
- Predict every unseen item with the weighted-sum formula, sort descending, and take the top N — that sorted slice is the recommendation list.
Every neighborhood-based recommender (and libraries like Surprise's KNNWithMeans) lets you choose cosine, Pearson, or adjusted cosine — and the top-N step is the final ranking users actually see.
If you remove it: Without bias-corrected similarity, neighborhoods fill with users who share rating habits, not taste, and the top-N list confidently recommends the wrong items.