11 · TF-IDF in practice: end-to-end pipeline
Build a real text classifier — TF-IDF + LogisticRegression in a sklearn Pipeline, inspect feature importance, and tune with GridSearchCV.
Build a real text classifier: TF-IDF + LogisticRegression in a sklearn Pipeline — the workhorse for any tabular text task.
Without this:
Stuck explaining TF-IDF in the abstract without ever running it on real text.
The previous lesson gave you the math. Now let's build something real. A TF-IDF + LogisticRegression Pipeline is the first thing any NLP practitioner reaches for when classifying text — it runs in milliseconds, is fully interpretable, and often outperforms more complex models on small datasets.
The key steps are:
- TF-IDF vectorization: convert raw text into a sparse matrix where each column is a vocabulary token weighted by TF-IDF.
- Train a classifier: LogisticRegression works exceptionally well on sparse TF-IDF features because the decision boundary is a hyperplane in the token-weight space.
- Inspect feature importance: the learned coefficients map directly to token weights — positive coefficients mean the token pushes toward the positive class, negative toward the negative class. This gives you a fully interpretable model.
- Tune with GridSearchCV: the most impactful parameters are
ngram_range,min_df,sublinear_tf, and the regularisation strengthC.
By the end of this lesson you will have a pipeline that trains, evaluates, tunes, and explains itself — the baseline every more complex model must beat.
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
TF-IDF tuning cheat sheet — the four parameters that matter most
What is the typical role of `min_df=5` in `TfidfVectorizer`?
- A `Pipeline([('tfidf', TfidfVectorizer(...)), ('clf', LogisticRegression())])` is the production-safe workhorse for text classification — one artifact to save, zero train/serve skew.
- LogisticRegression coefficients are directly interpretable as token importance — map them to feature names via `pipe.named_steps['tfidf'].get_feature_names_out()` to see which tokens drive each class.
- TF-IDF tuning priority order: `ngram_range=(1,2)` first, then `min_df=5` for noise reduction, then `sublinear_tf=True` for long documents, then `clf__C` for regularisation.
- Always run TF-IDF + LogReg as your baseline before investing in embeddings — it beats fine-tuned BERT on tiny datasets and explains itself completely via coefficient inspection.
Wikipedia search, internal docs search, spam filters in the pre-transformer era — all were TF-IDF + linear-model pipelines. Modern hybrid retrievers combine TF-IDF sparse retrieval with embedding-based dense retrieval via reciprocal rank fusion (RRF), giving the best of both worlds: exact keyword matching and semantic similarity.
If you remove it: Without a working TF-IDF pipeline, you have no interpretable baseline to benchmark embeddings against. The most common NLP mistake is going straight to a transformer without ever establishing whether the simpler model is already good enough.