13 · Top-k ranking metrics
Precision@k, recall@k, hit-rate, MAP, and NDCG each reward something different. NDCG cares not just whether the right item is in the list, but how high.
Top-k metrics turn a ranked list into a number. Precision@k and recall@k ask how many relevant items made the top k; hit-rate asks did ANY; MAP and NDCG go further and reward ranking the relevant items HIGH — NDCG discounting each hit by the log of its position.
Without this:
Optimize plain accuracy and you can't tell a list that buries the one good item at position 10 from one that puts it at position 1 — even though users only look at the top.
Once you hold out relevant items per user (last lesson), you score a recommender by how well its ranked top-k list recovers them. Each metric encodes a different question:
- Precision@k — of the k items I recommended, what fraction were relevant? Rewards not wasting slots. (k in the denominator.)
- Recall@k — of all the items that were relevant, what fraction did I surface in my top k? Rewards coverage of the truth. (relevant-count in the denominator.)
- Hit-rate@k — binary: did at least one relevant item land in the top k? The lenient leave-one-out favorite (with one held-out item, hit-rate@k = recall@k).
- MAP (Mean Average Precision) — averages precision each time you hit a relevant item as you walk down the list, then averages over users. Rewards putting relevant items early and clustering them.
- NDCG (Normalized Discounted Cumulative Gain) — sums a gain for each relevant item, discounted by
1 / log2(position + 1)so a hit at rank 1 is worth far more than at rank 10, then normalizes by the best-possible (ideal) ordering so the score lands in [0, 1].
The crucial divide: precision/recall/hit-rate are set metrics — they only care whether a relevant item is in the top k, not where. MAP and NDCG are rank-aware — moving a relevant item up the list, even within the top k, improves them. Since users overwhelmingly click near the top, rank-aware metrics correlate best with real engagement.
Below we implement all four from scratch and score two ranked lists against the same held-out relevant set. Then we move one relevant item from rank 4 up to rank 1 and watch NDCG (and MAP) rise while precision@k stays flat — the whole point of a position-discounted metric.
Python (in browser)
precision@k, recall@k, hit-rate, MAP and NDCG from scratch. List B ranks the same relevant items higher — set metrics stay flat, but MAP and NDCG climb.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
scikit-learn ships ndcg_score; full recsys frameworks (RecBole, Surprise) report precision/recall/NDCG/hit-rate over all held-out users for you.
You move a held-out relevant item from rank 4 up to rank 1 within your top-5 list. Which metric is GUARANTEED to improve?
- Precision@k, recall@k and hit-rate are SET metrics — they only care whether a relevant item is in the top k, not where.
- MAP and NDCG are RANK-AWARE: NDCG discounts each hit by 1/log2(rank+1), so higher positions score more.
- Report NDCG@k as the headline metric — it tracks real top-of-list engagement better than set metrics.
NDCG is the standard ranking metric in search and recsys leaderboards (MovieLens, Amazon datasets, the RecSys challenge); precision@k/recall@k headline most papers' results tables.
If you remove it: Without rank-aware metrics you optimize set membership and ship lists that technically contain the good item but bury it where nobody scrolls.