25 · pathlib: filesystem paths the modern way
Path objects beat string concatenation — join with /, inspect .stem / .suffix / .parent, glob for dataset shards, and read/write files in one method call.
`pathlib.Path` turns file paths from dumb strings into **smart objects** with attributes (`.name`, `.stem`, `.suffix`, `.parent`) and methods (`.exists()`, `.glob()`, `.mkdir()`, `.read_text()`). The `/` operator joins path segments: `Path("data") / "raw" / "train.csv"` — no more `os.path.join("data", "raw", "train.csv")`.
Without this:
Without `pathlib`, you join paths with string concatenation or `os.path.join`, extract file extensions with `os.path.splitext`, and check existence with `os.path.exists` — four different functions for what `Path` exposes as a single coherent object.
Before pathlib, the only way to manipulate file paths in Python was as plain strings using the os.path module: os.path.join("data", "raw", "train.csv"), os.path.splitext("train.csv")[0], os.path.dirname("data/raw/train.csv"). This works, but it scatters logic across many function calls and returns bare strings with no behaviour attached. pathlib.Path (introduced in Python 3.4, idiomatic since 3.6) gives you a single object that knows everything about itself. The / operator overrides the division operator to perform OS-correct path joining — Path("data") / "raw" / "train.csv" on Windows produces data\raw\train.csv automatically. Path objects are accepted by open(), pandas.read_csv(), torch.load(), and virtually every I/O function in the scientific stack.
Python (in browser)
`.parents` is a sequence of all ancestor paths from immediate parent up to the root. `.stem` strips the final suffix — `Path("archive.tar.gz").stem` gives `"archive.tar"` (only the last suffix is removed; use `.suffixes` for all of them).
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
`.write_text()` and `.read_text()` are convenience wrappers around `open()` — they handle file open, write/read, and close in a single method call. They don't replace `open()` for streaming or binary files, but they're perfect for small config files and log snippets.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
`.mkdir(parents=True, exist_ok=True)` is the safe idiom: `parents=True` creates every missing ancestor directory in the chain (like `mkdir -p` in bash); `exist_ok=True` suppresses the `FileExistsError` if the directory already exists. `.iterdir()` yields `Path` objects for every entry in the directory — files and sub-directories alike.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
`**/*.parquet` is a recursive glob — `**` matches zero or more directory levels. `.glob()` is lazy (it's a generator), so wrapping in `sorted()` or `list()` materialises it. In real ML projects `for p in Path("data").glob("**/*.parquet"):` is the canonical way to enumerate sharded datasets.
What does `Path("data/file.csv").stem` return?
- `from pathlib import Path` — use `/` to join, `.stem` / `.suffix` / `.name` / `.parent` to inspect. Replaces all `os.path.*` string functions.
- `.mkdir(parents=True, exist_ok=True)` creates a full directory tree safely. `.write_text()` / `.read_text()` are one-liner file I/O shortcuts.
- `Path("data").glob("**/*.csv")` finds all CSVs recursively. `Path` objects are accepted by `open()`, pandas, torch, and virtually every I/O function.
Every modern ML codebase uses `pathlib` for run directories: `run_dir = Path("runs") / experiment_name / run_id`. Checkpoint paths: `ckpt_path = run_dir / "checkpoint.pt"`. Dataset enumeration: `for csv_path in Path("data").glob("*.csv"):`. The HuggingFace Datasets library, PyTorch Lightning, and MLflow all accept `Path` objects in their save/load APIs.
If you remove it: Without `pathlib`, path construction in ML scripts devolves to fragile string concatenation — which breaks on Windows (`\` vs `/`) and has no `.exists()` or `.glob()` built in.