22 · Packages: directories, __init__.py, relative imports
A package is a directory of modules. Learn how __init__.py organises the public API, how absolute and relative imports work inside a package, and how pip install -e . wires your experimental code into Jupyter.
A **package** is a directory that contains an `__init__.py` file (making Python treat it as a module namespace) plus any number of sub-modules and sub-packages. `from mypkg.utils import normalize` lets you build large, organised codebases where each concern lives in its own file.
Without this:
Without packages, every ML project is a flat pile of `.py` files. Once you have more than a handful, name collisions and circular imports become inevitable. Packages give you a hierarchical namespace: `myproject.data`, `myproject.models.transformer`, `myproject.training`.
A package is a directory with an __init__.py file. Python 3.3+ also supports namespace packages — directories without __init__.py — but an explicit __init__.py is still best practice because it lets you control what from mypkg import * exposes and gives readers a clear signal that the directory is a package. Nested packages are just subdirectories that also have __init__.py. Inside a package, you can use relative imports — from .utils import normalize — which are relative to the current module's position and survive package renames. From outside the package, use absolute imports: from mlproject.data import load_dataset. You can turn your own package into an installable library with a minimal pyproject.toml and pip install -e ., which creates an editable install — a symlink so your changes are immediately reflected without reinstalling.
`__init__.py` has two jobs: (1) tell Python this directory is a package, and (2) optionally re-export names so callers use `from mlproject import load_dataset` instead of the longer `from mlproject.data import load_dataset`. Keep `__init__.py` thin — re-export key public names, leave the implementation in the submodules.
Absolute imports spell out the full path from the package root. They work from anywhere on `sys.path` and make the dependency structure explicit — a new reader can immediately see where `LinearModel` lives.
`pyproject.toml` replaced the older `setup.py` / `setup.cfg` as the standard way to declare a package's metadata and dependencies. The `requires-python` field protects users who still have Python 3.9.
What are the two main things an (empty) `__init__.py` file accomplishes?
- A package = a directory + `__init__.py`. Nested packages are subdirectories with their own `__init__.py`. Python 3.3+ allows namespace packages (no `__init__.py`) but explicit is better.
- From outside a package, use absolute imports: `from mlproject.data import load_dataset`. From inside, prefer relative imports: `from .utils import normalize`, `from ..data import load_dataset`.
- `pip install -e .` with a `pyproject.toml` creates an editable install — your source changes are immediately importable in notebooks and other scripts without reinstalling.
Every reusable ML repo is a package: `import myproject.training`, `import myproject.models.transformer`, `import myproject.data.loaders`. Without packages you're stuck with single-file scripts that can't share code without copy-pasting. Research repos like HuggingFace Transformers, PyTorch Lightning, and FastAI all expose their functionality via deep package hierarchies — `from transformers.models.bert import BertModel`.
If you remove it: Without packages, a project with separate data-loading, preprocessing, modelling, and evaluation concerns would need every function in a single monolithic file — or rely on fragile relative `sys.path` hacks. Package hierarchies are how ML teams maintain codebases that multiple contributors edit simultaneously.