49 · Multi-logger configuration: handlers, formatters, dictConfig
One logger, multiple destinations — attach a StreamHandler for the terminal and a FileHandler for persistent logs, each with its own level and format. Then replace manual wiring with logging.config.dictConfig for the production-ready, declarative approach.
A `Logger` is a named dispatcher — it knows nothing about *where* records go. `Handler`s decide the destination (terminal, file, HTTP endpoint); `Formatter`s decide the shape. Attaching multiple handlers to one logger means the same record is delivered to several places simultaneously — a critical capability for any pipeline that runs unattended. `logging.config.dictConfig({...})` lets you declare the entire logging tree in a single dictionary at startup, making the config auditable, version-controllable, and environment-swappable without touching application code.
Without this:
Without multi-handler configuration you are forced to choose: either you see logs in the terminal (and they vanish when the process ends) or you write to a file (and lose real-time visibility). `dictConfig` lets you have both, plus different formats and verbosity levels per destination, declared once and shared across the whole program.
Why multiple handlers?
In the previous lesson every log record went to one place — the terminal (stderr) via the root logger's default StreamHandler. Real pipelines need more:
- Terminal (
StreamHandler) — human inspection during development. You usually only wantWARNINGand above so the console stays readable. - File (
FileHandler/RotatingFileHandler) — fullDEBUGoutput written to disk for post-mortem analysis. - Remote (
HTTPHandler,SocketHandler, or a third-party sink) — forwarded to a log aggregator like Datadog, Elastic, or CloudWatch.
Each handler has its own level (a second filter after the logger's own level) and its own Formatter (the string template). A record must pass both the logger level and the handler level to be emitted.
The filtering chain
message is logged
│
▼
Logger level ──── too low? ──► silently dropped
│
▼ (for each attached Handler)
Handler level ──── too low? ──► skipped for this handler
│
▼
Formatter → final string → destination
Three ways to configure logging
- Manual API —
logger.addHandler(handler)— useful for understanding the model; awkward at scale. logging.config.dictConfig(d)— declare the full config as a Python dict; the idiomatic production approach.logging.config.fileConfig(path)— older INI-style file; less expressive, rarely used in new code.
Python (in browser)
The console prints only `WARNING` and `ERROR` — the two records that cleared both the logger level (`DEBUG`) and the handler level (`WARNING`). The file contains all four because its handler level is `DEBUG`. Same logger, same `logger.debug(...)` call, different destinations.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
`dictConfig` produces identical behaviour to the manual API — the difference is operational: the config dict can live in a YAML/JSON config file, be overridden per environment (dev/staging/prod), and be read by DevOps tooling without touching Python source. `"propagate": False` prevents the record from also reaching the root logger (which would cause double-printing).
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
The `NullHandler` convention is enforced in popular libraries like `requests`, `boto3`, and `httpx` — they all declare `logging.getLogger(__name__).addHandler(logging.NullHandler())` and leave routing decisions entirely to the application.
You configure a logger with `logger.setLevel(logging.DEBUG)` and attach a single `StreamHandler` with `handler.setLevel(logging.WARNING)`. You then call `logger.info("checkpoint saved")`. What happens?
- A `Logger` can have multiple `Handler`s — each with its own level and `Formatter`. A record must pass both the logger level and the handler level to be emitted.
- Built-in handlers: `StreamHandler` (stderr), `FileHandler` (file), `RotatingFileHandler` (auto-rotate on size), `TimedRotatingFileHandler` (auto-rotate on time).
- `logging.config.dictConfig({...})` is the idiomatic production way to configure the full logging tree: formatters, handlers, and loggers declared once in a single dict.
- Library authors: only ever add `logging.NullHandler()` to your logger. Never call `basicConfig` or add real handlers — that is the application entry point's job.
- Set `"propagate": False` in `dictConfig` loggers to prevent records from travelling up to the root logger and being printed twice.
Production ML services use multi-handler configurations as a matter of course: a `StreamHandler` at `WARNING` for human inspection in the terminal during development, a `StreamHandler` at `DEBUG` emitting JSON to stdout for cloud log aggregation (Cloud Logging, CloudWatch, Datadog), and a `RotatingFileHandler` for high-volume debug data that is archived to S3 or GCS. The `dictConfig` dictionary usually lives in `config/logging.yaml`, loaded at startup — one change and every logger in the application follows the new rules without a code deploy.
If you remove it: Without multi-handler config you're forced to choose one destination per script run. Debug sessions require changing code (adding a FileHandler manually, then removing it), and there is no clean way to have the terminal show only warnings while the file captures full debug detail. At scale this means either noisy terminals or blind spots in your logs.