7 · Bag of Words (BOW)
Turn any document into a fixed-size count vector with CountVectorizer — the entry ticket to sklearn text classification.
BOW represents a document as the COUNT vector of its tokens — one row, vocab-size columns, ignoring word order.
Without this:
You can't run sklearn's LogisticRegression or NaiveBayes on text without converting documents to fixed-size vectors first.
One-hot encoding gave each word its own dimension — but left us with variable-length sequences and extreme sparsity. Bag of Words (BOW) fixes the variable-length problem with one elegant move: instead of keeping each word as a separate OHE row, we sum all the OHE rows into a single vector of word counts.
The result is a count vector of length equal to the vocabulary size. Every document — regardless of length — is represented as a single row with vocab-size columns. Column i holds the number of times word i appeared in that document.
Why is it called a "bag"? Because you throw all the words in a bag and count how many times each word appears. Word order is completely discarded. "Dog bites man" and "Man bites dog" have identical BOW vectors — a real limitation for sequence-sensitive tasks, but fine for many classification scenarios.
sklearn's CountVectorizer handles the entire pipeline: tokenization, vocabulary construction, sparse matrix building. It's the go-to first tool for text classification.
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
BOW pros and cons — when to reach for it and when not to
What does `CountVectorizer().fit_transform(docs).shape[1]` represent?
- `CountVectorizer().fit_transform(docs)` returns a sparse matrix of shape `(n_docs, vocab_size)` where each cell is the count of that word in that document.
- Use `stop_words='english'` and `max_features=N` to keep vocabulary manageable and remove high-frequency noise words.
- BOW discards word order entirely — 'dog bites man' and 'man bites dog' produce the same vector. For tasks where order matters (sentiment with negation), n-grams or embeddings are needed.
- Never call `.toarray()` on large sparse matrices — work with sklearn classifiers that accept sparse input directly.
Spam classifiers, sentiment classifiers, topic models (LDA), document clustering — every classical NLP system starts with BOW or TF-IDF. sklearn's text classification examples in the official docs almost always use CountVectorizer.
If you remove it: Without BOW you cannot feed raw text to any sklearn classifier — LogisticRegression, SVM, RandomForest, and NaiveBayes all require numeric feature matrices.