19 · Map, filter, reduce
Lazy iterators for transforming and filtering sequences — and when a comprehension is the more Pythonic choice.
`map(fn, iterable)` and `filter(pred, iterable)` return **lazy iterators** — they produce values one at a time only when requested, so they never build the full output list in memory. `functools.reduce` folds a sequence into a single value.
Without this:
Without lazy iteration, transforming a 1-million-sample dataset in one step allocates a full copy in memory. `map` and `filter` let you chain transforms that stream data without materializing intermediate collections.
map(function, iterable) applies function to every element and returns a map object (a lazy iterator). Wrap it in list() to materialize the result. filter(predicate, iterable) keeps only elements for which the predicate returns truthy. Both are lazy — they don't evaluate anything until you iterate them. functools.reduce(function, iterable, initial) cumulatively applies a two-argument function to collapse the sequence into a single value. For most day-to-day transforms and filters, a list comprehension is more idiomatic — but map and filter shine in pipelines where you want laziness and composability.
Python (in browser)
`map()` returns an iterator, not a list. Calling `next()` on it pulls the next transformed value. This laziness means you can safely `map` over a million-element dataset without copying it.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
`filter(None, iterable)` is a quick way to drop falsy values (0, empty string, None, empty list). Chaining `filter` then `map` composes lazily — the full pipeline runs on demand.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
`reduce` applies the function cumulatively: `reduce(f, [a,b,c], init)` = `f(f(f(init, a), b), c)`. For `sum`, `max`, and `min` the built-ins are faster and more readable. Reserve `reduce` for non-standard folds like dict merging.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
What type does `map(str.upper, ['a', 'b'])` return?
- `map(fn, iter)` and `filter(pred, iter)` return lazy iterators — wrap with `list()` to materialise. They compose cheaply without copying data.
- `functools.reduce(fn, iter, init)` folds a sequence into one value. Use built-ins (`sum`, `max`, `min`) for the common cases — `reduce` for custom folds.
- For simple transforms/filters, a list comprehension is more idiomatic and readable. Prefer `map`/`filter` when laziness or an existing named function makes them the cleaner choice.
Lazy preprocessing pipelines: `map(preprocess, raw_samples)` streams a dataset through a transform without loading it all into memory. Dropping invalid rows: `filter(is_valid, dataset)`. Summing gradients across workers: `reduce(np.add, worker_gradients)`. In PyTorch, `DataLoader` workers apply transforms lazily — the same `map`-over-iterator model.
If you remove it: Without lazy iteration, a preprocessing pipeline that chains several transforms must materialise an intermediate list at each step — for a dataset of 500 000 images this could mean gigabytes of unnecessary copies before a single batch reaches the model.