4 · One-hot encoding for nominal features
Nominal categories (no order) become one-hot vectors — N categories ⇒ N (or N−1) binary columns.
Nominal categories (no order) become one-hot vectors — N categories ⇒ N (or N−1) binary columns.
Without this:
Without it, models receive arbitrary integer codes that pretend 'apple=1' is closer to 'banana=2' than to 'mango=5' — which is meaningless.
Before any model can learn from a categorical feature like color, it needs numbers. The key insight is whether the categories have a natural order.
- Nominal features have NO order:
color = {red, green, blue},country = {US, MX, CA}. There is no sense in which "red < green < blue". - Ordinal features DO have an order:
size = {S, M, L, XL},rating = {low, medium, high}.
For nominal features, the standard transformation is one-hot encoding (OHE): create one binary column per category, where the column is 1 if the row belongs to that category and 0 otherwise.
A row with color = "red" becomes:
| color_red | color_green | color_blue | |-----------|-------------|------------| | 1 | 0 | 0 |
This preserves the equality between categories — "red" is neither closer to nor further from "green" than from "blue" — they are orthogonal directions in feature space.
Python (in browser)
`pd.get_dummies` creates one binary column per unique category, named `{prefix}_{category}` in alphabetical order. Each row has exactly one 1.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
`drop_first=True` removes one column — the third is implied when the other two are 0. Matters for linear models (avoids multicollinearity); irrelevant for tree models.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
`OneHotEncoder` from sklearn fits on training data and applies the SAME schema to test data. `handle_unknown='ignore'` silently maps unseen categories to all-zeros — critical for production systems.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
The dummy variable trap: N one-hot columns sum to 1 ≡ the intercept. Drop one to break the perfect collinearity. Only matters for linear models (logistic regression, linear regression, SVM with linear kernel).
If a column has categories {A, B, C, D} and you use `pd.get_dummies(drop_first=True)`, how many columns are produced?
- Nominal features have no natural order — one-hot encoding maps each category to its own binary column, preserving category equality.
- `pd.get_dummies` is fine for EDA; `sklearn.preprocessing.OneHotEncoder` is required for production — it has `.fit()` / `.transform()` and handles unseen categories via `handle_unknown='ignore'`.
- Use `drop_first=True` (or `drop='first'` in sklearn) for linear models to avoid the dummy variable trap (multicollinearity). Not needed for tree models.
- High-cardinality categoricals (1000+ categories) blow up dimensionality with OHE — use target encoding or embeddings instead.
Every classification or regression with a categorical feature begins with encoding. `ColumnTransformer` in sklearn applies OHE to nominal columns and a different transformer to numeric columns — the standard backbone of any sklearn Pipeline.
If you remove it: Linear models can't ingest strings; tree models can but most APIs (XGBoost, LightGBM in some configs) still require numeric input. Omitting encoding means `model.fit(X, y)` raises a TypeError before any learning happens.