33 · Iterators and the iterator protocol
Understand the two-method contract that powers every for loop — __iter__ and __next__ — and write your own custom iterator class.
An **iterable** is any object with `__iter__()` — a list, a string, a file. An **iterator** is the stateful cursor you get back from calling `iter(obj)`. It has `__next__()` which returns one value at a time and raises `StopIteration` when the sequence is exhausted. Every `for` loop is syntactic sugar for exactly this two-step dance.
Without this:
Without understanding iterators you can't build custom data sources that plug into Python's `for` loop, `zip()`, `enumerate()`, or any function that expects an iterable. Frameworks like PyTorch express `DataLoader` as an iterator — calling `iter(loader)` resets the cursor for a new epoch.
Python distinguishes between two related but different concepts:
- Iterable — any object that implements
__iter__(). A list, a string, a range, a file are all iterables. When you ask for their iterator you always get a fresh one. The iterable itself does not track position. - Iterator — the stateful cursor produced by calling
iter(obj). It implements two methods:__iter__()(which simply returnsself) and__next__()(which returns the next value or raisesStopIterationto signal the end).
The key rule: you can call iter() on a list over and over and get a fresh iterator each time. An iterator, by contrast, is write-once — once exhausted it stays exhausted. You can't rewind a raw iterator; you'd have to call iter(original_iterable) again.
Built-in shortcuts: iter(obj) is the same as calling obj.__iter__(), and next(it) is the same as it.__next__().
Python (in browser)
The iterator `it` is a separate object from the list `numbers`. After three `next()` calls the cursor is past the end. The list is unchanged — calling `iter(numbers)` again yields a brand-new iterator from the beginning.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
`__iter__` returns `self` because the `Countdown` instance is both the iterable and the iterator — it holds the state. This is the standard pattern for custom iterator classes.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Every `for` loop compiles down to exactly these six lines. Understanding this makes it obvious why an iterator only walks forward and why `StopIteration` is the signal — not an error — that the sequence is done.
Is a Python list an iterator?
- **Iterable** = has `__iter__()`; **Iterator** = has both `__iter__()` (returns self) and `__next__()` (returns next value or raises `StopIteration`).
- `for x in obj:` calls `iter(obj)`, then `next(...)` in a loop until `StopIteration` — no other magic involved.
- Lists are iterables but not iterators. Calling `iter([])` returns a fresh `list_iterator` every time — the original list is never mutated.
- Most raw iterators are write-once — after `StopIteration` they remain exhausted. To repeat, call `iter()` on the original iterable.
PyTorch's `DataLoader` is an iterator — each training epoch you call `iter(loader)` to get a fresh cursor over the shuffled dataset. `enumerate(loader)` wraps it to give `(batch_idx, batch)` pairs. Streaming data sources (HuggingFace `IterableDataset`, database cursors, network streams) are exposed as iterators because the full data doesn't fit in RAM — you must pull one chunk at a time.
If you remove it: Without the iterator protocol, every data pipeline would need to materialise the full dataset into a list before the first epoch starts. For ImageNet-scale or streaming datasets that is simply impossible — the iterator model is what makes lazy, memory-efficient training loops feasible.