1 · The NLP roadmap
A bird's-eye view of the field — from raw tokens to Transformers — and the six real-world use cases that motivate the whole track.
NLP turns text into vectors a model can learn — and the field's progression from tokens → BOW → TF-IDF → Word2Vec → Transformers IS the roadmap of this track.
Without this:
You'd jump into BERT without understanding what came before, missing why each layer matters.
Natural Language Processing (NLP) is the branch of ML that deals with human language. Text is one of the richest, most abundant data sources in the world — emails, reviews, news articles, code, legal contracts, medical notes — and yet raw text is completely opaque to most ML algorithms, which expect numbers.
The central challenge of NLP is: how do we turn words into numbers in a way that preserves meaning? Every technique in this track is an answer to that question, and each answer is progressively more powerful:
| Era | Representation | Key idea | |---|---|---| | Tokenization | raw tokens | Split text into discrete units | | BOW / Bag of Words | count vectors | Count token occurrences | | TF-IDF | weighted counts | Weight by rarity across documents | | Word2Vec | dense embeddings | Predict context, capture semantics | | Transformers | contextual embeddings | Attend to the whole sequence at once |
This track covers the first four eras in depth. Transformers belong to the Deep Learning track — but understanding the classical pipeline is the foundation that makes Transformers make sense.
The classical NLP pipeline — every real system has these stages
Six canonical NLP use cases — the same pipeline, different outputs
Which is the FIRST step in any NLP pipeline?
- NLP converts raw text into numeric vectors so that ML models can learn from language.
- The field progressed from simple token counts (BOW) → weighted counts (TF-IDF) → dense semantic vectors (Word2Vec) → contextual embeddings (Transformers).
- Tokenization is the universal first step — every downstream technique assumes text has already been split into tokens.
- Understanding the classical pipeline makes modern Transformer tools (Hugging Face, LangChain) far easier to use and debug.
Every NLP system you'll build or use starts from here — the pipeline you just mapped is the skeleton of every spam filter, sentiment classifier, chatbot, and search engine.
If you remove it: Without this roadmap you'd pick tools at random and miss the 'why' behind tokenization, embeddings, and attention — the three pillars of modern NLP.