27 · Chi-square test
Test whether observed counts match expected counts — for fitting a model (goodness of fit) or detecting association between two categorical variables (independence test).
Chi-square tests whether OBSERVED counts match EXPECTED counts — for either fitting a model or checking independence in a contingency table.
Without this:
Without it, you can't formally compare observed-vs-expected counts in any categorical analysis.
All the tests we've covered so far deal with continuous measurements (means, variances). But a lot of real-world data is categorical: gender, device type, treatment group, die faces, survey answers. The chi-square test (χ²) is the workhorse for categorical data.
The chi-square statistic measures how far observed counts stray from what you'd expect under some null model:
χ² = Σ (O − E)² / E
The sum runs over every cell in your frequency table. When every observed count matches its expected value exactly, χ² = 0. Larger values mean bigger deviations.
Under the null hypothesis, χ² follows a chi-square distribution with df degrees of freedom. The chi-square distribution is right-skewed and bounded below by 0 — naturally, since squared deviations are always non-negative.
Two flavours of chi-square test:
| Test | Question | df | |------|----------|-----| | Goodness of fit | Does this single categorical var follow a hypothesised distribution? | k − 1 | | Independence | Are two categorical vars independent in this contingency table? | (r−1)(c−1) |
The statistic formula and null-rejection rule are identical for both; only the source of the expected counts differs.
Python (in browser)
Goodness-of-fit test on 60 die rolls. We compute χ² manually first to make the formula concrete, then verify with scipy.stats.chisquare. df = k − 1 = 5.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Independence test on a 2×3 contingency table (treatment/control vs. improved/no-change/worse). scipy.stats.chi2_contingency returns the statistic, p-value, df, and the expected counts matrix.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Chi-square distributions for df = 1, 4, 10. As df grows, the distribution shifts right and becomes more symmetric. The shaded tail is the rejection region at α = 0.05.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Assumption check: if any expected count < 5, Fisher's exact test (for 2×2 tables) is the correct alternative. The odds ratio measures how strongly the two outcomes are associated.
For a chi-square goodness-of-fit test on a 6-sided die (categories: faces 1–6), what is the degrees of freedom?
- **Chi-square statistic:** χ² = Σ (O − E)² / E, summed over all cells. Large χ² → the observed counts are unlikely under the null.
- **Goodness of fit** (one categorical var): df = k − 1, where k is the number of categories.
- **Independence test** (two categorical vars, r×c table): df = (r − 1)(c − 1). Use `scipy.stats.chi2_contingency`.
- **Key assumption:** all expected cell counts ≥ 5. If any cell is < 5, switch to Fisher's exact test.
Categorical feature selection (`sklearn.feature_selection.chi2`); detecting train/production distribution shifts on categorical fields; A/B test conversions broken down by user segment.
If you remove it: You can't formally answer 'does this categorical feature have a relationship with the target?'