53 · concurrent.futures: ThreadPool and ProcessPool the modern way
concurrent.futures wraps raw threads and processes in a clean Future-based API. Swap ThreadPoolExecutor for ProcessPoolExecutor by changing one word, and use as_completed() to handle results as they finish.
`concurrent.futures` is Python's high-level concurrency API. `ThreadPoolExecutor` and `ProcessPoolExecutor` are drop-in interchangeable — the same `executor.submit()`, `executor.map()`, and `as_completed()` interface works for both. This symmetry means you can prototype with threads (no Pyodide restrictions, instant start) and switch to processes by changing one class name when profiling reveals the bottleneck is CPU. `as_completed(futures)` gives you results as they finish — not in submission order — which is exactly what you want for variable-latency I/O tasks.
Without this:
Without `concurrent.futures` you manage raw threads or processes manually — tracking which threads are still running, collecting their results, handling exceptions per-thread. This boilerplate is error-prone and makes switching between thread and process pools a multi-hour refactor. `concurrent.futures` reduces it to a one-word swap.
The concurrent.futures design
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor, as_completed
Three central concepts:
-
Executor — a pool manager.
ThreadPoolExecutor(max_workers=N)creates N threads;ProcessPoolExecutor(max_workers=N)creates N processes. Both are context managers (withcloses the pool gracefully). -
Future — a handle to a result that isn't ready yet.
future.result()blocks until the value is available;future.exception()returns any exception that was raised;future.done()is non-blocking. -
Two submission methods:
executor.submit(fn, *args)— submits one task, returns aFutureimmediately.executor.map(fn, iterable)— submits all tasks, returns results in submission order (lazy iterator; each.next()may block).
as_completed vs map
| | executor.map(fn, items) | as_completed(futures) |
|---|---|---|
| Order | Submission order | Completion order |
| Best for | Ordered pipelines | Variable-latency I/O |
| Exception handling | Raised when you iterate | Call future.exception() or future.result() |
The golden rule
Start with sequential. If I/O-bound →
ThreadPoolExecutor. If CPU-bound →ProcessPoolExecutor. Reach for rawthreading.Threadonly when you need fine-grained control (custom daemon threads, manual join order, per-thread lifecycle hooks).
Python (in browser)
`executor.map(fn, iterable)` is a direct drop-in replacement for Python's built-in `map(fn, iterable)` — with the executor distributing tasks across the pool. The context manager (`with`) calls `executor.shutdown(wait=True)` automatically: all submitted tasks complete before the block exits.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Locally, tasks 3 (1 ms) and 1 (2 ms) would finish and print first even though tasks 0–2 were submitted first. In Pyodide they finish in submission order because threads are sequential. The `future_to_id` dict is the standard pattern for mapping a `Future` back to its input.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
This pattern — `submit` all tasks first, then `as_completed` — is what scraping frameworks, parallel API clients, and parallel test runners are built on. The thread pool handles scheduling; you just process results as they arrive.
The one-word swap from `ThreadPoolExecutor` to `ProcessPoolExecutor` is the entire point of `concurrent.futures`. Prototype with threads (instant startup, no Pyodide restrictions), measure, then switch to processes if the profiler shows CPU saturation.
What does `as_completed(futures)` return?
- `concurrent.futures` provides a unified `Future`-based API for both thread pools (`ThreadPoolExecutor`) and process pools (`ProcessPoolExecutor`). Swapping between them requires changing one class name.
- `executor.submit(fn, *args)` returns a `Future` immediately. `executor.map(fn, iterable)` is a convenience wrapper that yields results in submission order.
- `as_completed(futures)` yields `Future` objects in **completion order** — fastest first. Ideal for variable-latency I/O tasks where you want to process results as they arrive.
- The context manager (`with executor:`) calls `executor.shutdown(wait=True)` automatically — all submitted tasks complete before the block exits.
- Reach for `concurrent.futures` first. Use raw `threading.Thread` only when you need fine-grained lifecycle control. `joblib.Parallel` is a higher-level wrapper with better numpy/ML defaults.
Parallel data preprocessing in a training pipeline — resizing and normalizing images before they enter a DataLoader — is a classic `ThreadPoolExecutor` workload when the bottleneck is disk reads (I/O). Batch inference on multiple independent inputs (scoring N documents simultaneously) maps naturally to `executor.map(score, documents)`. Parallel hyperparameter search (`GridSearchCV(n_jobs=-1)`) and `joblib.Parallel` in scikit-learn are thin wrappers over `ProcessPoolExecutor`. Running multiple experiment configurations simultaneously on a multi-GPU machine uses process-level isolation to prevent shared GPU state.
If you remove it: Without `concurrent.futures`, switching a preprocessing pipeline from threads to processes to find which is faster requires a significant refactor — rewriting `Thread` + manual join logic into `Process` + manual join logic. With executors, the same code serves both, and you can benchmark the switch in 30 seconds. This is the primary reason `concurrent.futures` exists: to make concurrency mode a configuration detail, not an architecture decision.