5 · Choosing K: elbow, silhouette, k-means++
Two diagnostics tell you K: the elbow in inertia vs K, and the silhouette score peak — plus why k-means++ gives you a head start.
Two diagnostics tell you K: the elbow in inertia vs K, and the silhouette score peak.
Without this:
Without these, picking K is pure guesswork.
K-Means requires you to specify K upfront — but real-world data rarely tells you how many clusters to expect. Two classic diagnostics help you choose:
1. The elbow method plots inertia vs K. As K increases, adding a new cluster always reduces inertia (in the extreme, K = N gives zero inertia). The idea: look for the elbow — the K beyond which adding more clusters stops reducing inertia much. It is a heuristic, not a theorem, but it is fast and often useful.
2. Silhouette score — for each point i, compute:
s(i) = (b(i) − a(i)) / max(a(i), b(i))
where a(i) = mean distance to all other points in the same cluster (cohesion), and b(i) = mean distance to all points in the nearest other cluster (separation). Silhouette ranges from −1 to 1:
- +1 → point is perfectly in its own cluster and far from neighbours
- 0 → point is on the boundary between two clusters
- −1 → point is closer to another cluster than its own (likely misassigned)
The mean silhouette score across all points is a summary quality metric — higher is better. Plot it vs K and pick the peak.
3. K-Means++ initialization — Instead of fully random starting centroids, k-means++ picks the first centroid uniformly at random, then picks each subsequent centroid with probability proportional to the squared distance from the nearest existing centroid. This deliberately spreads the initial centroids far apart, giving Lloyd's algorithm a much better starting point.
Two additional metrics to cross-check your choice:
- Davies-Bouldin (DB) index — ratio of within-cluster scatter to between-cluster separation; lower is better.
- Calinski-Harabasz (CH) index — ratio of between-cluster to within-cluster dispersion; higher is better.
Python (in browser)
Elbow plot — inertia decreases sharply up to K=5 and flattens afterward. The red dashed line marks the true K.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Silhouette score peaks at K=5 — this agrees with the elbow plot and the true number of clusters.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
20 trials comparing random vs k-means++ initialization — k-means++ consistently achieves lower mean inertia with less variance.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
All three metrics agree at K=5 — silhouette peaks (higher is better), Davies-Bouldin is near minimum (lower is better), Calinski-Harabasz peaks (higher is better).
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Decision guide — when elbow, silhouette, DB, and CH disagree on the best K.
If the silhouette score for a clustering is 0.05, the clustering quality is...
- The elbow method plots inertia vs K — look for the bend where adding clusters stops paying off in compactness.
- Silhouette score = (b−a)/max(a,b) ranges from −1 to +1; higher is better; pick the K with the highest mean silhouette.
- K-Means++ spreads initial centroids far apart (proportional to squared distance), consistently reaching better solutions than pure random initialisation.
- Davies-Bouldin (lower is better) and Calinski-Harabasz (higher is better) provide additional cross-checks; when diagnostics disagree, lean on silhouette and visual inspection.
- K-Means++ has been sklearn's default since v0.23 — you get it automatically without specifying init.
Customer-segment count selection in marketing analytics; topic-count selection in unsupervised NLP (LDA topic models); cluster count in dimensionality-reduction-then-cluster pipelines.
If you remove it: You'd default to K=3 or K=5 by superstition.