23 · Project: RNN sentiment classifier (IMDB)
Build an end-to-end RNN sentiment classifier on a synthetic IMDB-style dataset: vocabulary, embedding lookup, RNN cell, classification head — all in NumPy.
Build an end-to-end RNN sentiment classifier on a tiny synthetic IMDB-style review dataset — bringing together embeddings, RNN, and a classification head.
Without this:
Without the project, RNN concepts stay abstract — you know the equations but haven't assembled them into a working pipeline.
This lesson is a synthesis project: we'll build every piece of a real NLP pipeline from scratch, in NumPy, starting from raw text and ending with predicted sentiment labels.
The pipeline has four stages that mirror every pre-transformer NLP model ever built:
- Tokenisation + vocabulary — map words to integer indices.
- Embedding lookup — convert integer indices to dense float vectors (the embedding matrix is just a look-up table; in practice it's learned during training, like GloVe or Word2Vec).
- RNN encoder — process the sequence of embedding vectors with the RNN cell; the final hidden state
h_Tis the "sentence vector." - Classification head — a single linear layer + sigmoid maps the sentence vector to a positive/negative probability.
We'll use a 40-review synthetic IMDB-style dataset to keep compute times fast inside Pyodide. The vocabulary is tiny (~50 words), so the model won't reach production accuracy — but every line of code directly maps to a real PyTorch SentimentRNN module shown at the end.
This is the same architecture that Socher et al. (2013) used on the Stanford Sentiment Treebank, and that Google's SmartReply used (with LSTMs) in production until 2019.
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
PyTorch read-along: full SentimentRNN with Embedding, RNN, and classification head (~30 lines)
The 'real' version: how production NLP sentiment classifiers differ from our hand-rolled model
Why is the embedding layer essential in an RNN sentiment classifier?
- The Embedding → RNN → Linear head pipeline is the foundation of every pre-transformer NLP model.
- Embeddings convert sparse one-hot indices to dense semantic vectors; pre-trained embeddings (GloVe) are essential for small datasets.
- The final hidden state h_T is the 'sentence vector' in a many-to-one classification setup.
Every pre-transformer NLP model (translation, NER, intent classification, chatbot intent detection) followed this exact Embedding → RNN → Output template. Hugging Face's encoder models simply replaced the RNN with a transformer encoder — the embedding lookup and output head remain nearly identical.
If you remove it: You'd use a transformer fine-tuning recipe without understanding the embedding lookup that every LLM still uses internally, or the sentence-vector pattern that powers bi-encoder semantic search (e.g. FAISS + Sentence-BERT).