23 · Useful standard library modules
A focused tour of datetime, pathlib, json, collections, itertools, functools, and random — the stdlib modules every ML practitioner reaches for before pip-installing anything.
Python's standard library is 'batteries included' — `datetime`, `pathlib`, `json`, `itertools`, `functools`, and `random` are all built-in, zero-install, and cover most of what an ML script needs before it ever touches NumPy or pandas.
Without this:
Without the stdlib, even reading a JSON config file, constructing a file path, or seeding a random number generator would require a third-party library. Knowing the stdlib saves you from over-engineering simple tasks.
Python ships with a rich standard library — no pip install needed. This lesson tours the modules that show up most in ML and data-science work: datetime for timestamps and experiment naming; pathlib for modern cross-platform path handling; json for serialising configs and results; collections for specialised containers (Counter, defaultdict, deque); itertools for memory-efficient combinatorial iteration; functools for memoisation and partial application; and random for reproducible stochastic sampling. Each example is self-contained and runnable in Pyodide.
Python (in browser)
`datetime.now().isoformat()` gives an unambiguous, sortable timestamp. Strip colons with `strftime` to use it in file paths — `runs/exp-20240315_142207/` is valid on every OS.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
The `/` operator on `Path` objects joins path segments — far more readable than `os.path.join("data", "raw", "train.csv")`. `Path` objects work directly with `open()`, `glob()`, `mkdir()`, and most file I/O functions.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
`json.dumps` (string) and `json.dump` (file) serialise Python dicts, lists, strings, numbers, and booleans. `indent=2` produces human-readable output. JSON is the default config format in most ML tools — Hugging Face `config.json`, PyTorch Lightning `hparams.yaml` can also be round-tripped via JSON.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
`itertools.product(*grids)` is how you build a hyperparameter sweep grid without nested loops. `combinations` generates all unique feature pairs for interaction terms. Both are lazy — they never build the full list in memory.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
`partial(fn, **fixed_kwargs)` returns a new callable with some arguments pre-filled — perfect for baking in fixed preprocessing parameters like mean/std so a transform function has the right signature for `map()`. `lru_cache` memoises a function's results by argument value.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
`random.seed(42)` makes the sequence reproducible across runs — the exact same numbers every time. NumPy and PyTorch have their own RNG states (`np.random.seed(42)`, `torch.manual_seed(42)`); you usually need to seed all three in an ML script.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
What does `Counter("mississippi").most_common(2)` return?
- `datetime.now().isoformat()` — unambiguous timestamps for experiment dirs. `strftime("%Y%m%d_%H%M%S")` strips colons for safe file paths.
- `pathlib.Path` over `os.path` — use `/` to join, `.parent` / `.stem` / `.suffix` to inspect, `.mkdir(parents=True, exist_ok=True)` to create. Works directly with `open()`.
- `itertools.product(*grids)` builds hyperparameter search grids lazily. `functools.partial` freezes arguments. `random.seed(n)` makes sampling reproducible.
Timestamped experiment dirs: `Path("runs") / f"exp-{datetime.now().strftime('%Y%m%d_%H%M%S')}"`. JSON config files read with `json.load(open("config.json"))`. `pathlib.Path.glob("data/**/*.csv")` in data loaders. `random.seed(42)` paired with `np.random.seed(42)` and `torch.manual_seed(42)` for full reproducibility. `itertools.product(lr_values, batch_sizes)` for grid search. `partial(normalize, mean=dataset_mean, std=dataset_std)` to bake preprocessing stats into a reusable transform callable.
If you remove it: Without these stdlib modules, ML scripts would need third-party libraries for basic file-path manipulation, JSON serialisation, and random seeding — or hand-roll their own. The stdlib's stability also means these tools work identically across Python versions and environments, including Pyodide.