7 · Containerizing the API
Now wrap the FastAPI service in a Docker image: run it with uvicorn, expose the port, read the model path from an env var, and add a `/healthz` endpoint so the platform knows when the container is actually ready to serve.
Containerize the FastAPI service by running it with `uvicorn` as the container's `CMD`, `EXPOSE`-ing its port, and reading the model path from an ENV var. Add a `/healthz` endpoint so the orchestrator can tell LIVENESS (is the process up?) from READINESS (is the model loaded and able to serve?) — and route traffic only when ready.
Without this:
Without a health/readiness endpoint, the platform sends real user traffic to a container the instant the process starts — before the model has finished loading — so the first requests fail; and without env-driven config the same image can't be reused across environments.
You have a FastAPI app (lesson 6) and you know how to write a Dockerfile (lessons 3–5). Now combine them: package the API as a container image so it can be deployed anywhere.
Run it with uvicorn. A FastAPI app is served by an ASGI server — uvicorn. So the container's CMD launches uvicorn pointing at your app object: uvicorn app:app --host 0.0.0.0 --port 80. The --host 0.0.0.0 is essential — it tells uvicorn to listen on all interfaces, not just localhost; without it, requests from outside the container can't reach it. EXPOSE 80 documents which port the container serves on.
Env-driven config. Recall the FastAPI app read MODEL_PATH from an environment variable. That pays off here: the same image can point at different model files in dev, staging, and prod just by setting MODEL_PATH differently at run time. Configuration lives outside the image; only code + dependencies are baked in.
Health checks — liveness vs readiness. This is the key new idea. When an orchestrator (Kubernetes, a load balancer, a cloud platform) runs your container, it needs to answer two different questions:
- Liveness — is the process alive? If the process has hung or crashed, restart the container. This just checks the server is responding at all.
- Readiness — is this container ready to receive real traffic? A model API has slow startup: the process is up within milliseconds, but loading the
.joblibmodel takes longer. During that window the process is live but not ready — if traffic arrives now, requests fail because the model isn't loaded yet. Readiness says "don't route requests to me until I signal I'm ready."
You expose a lightweight /healthz endpoint the platform polls. A robust readiness check returns 200 OK only once the model is actually loaded (e.g. model is not None); before that it returns a not-ready status, and the orchestrator holds traffic back. This is why the first requests to a freshly started, un-instrumented model container so often fail — and why a readiness endpoint fixes it.
A Dockerfile for the FastAPI service: install deps, env-driven `MODEL_PATH`, `EXPOSE 80`, and a uvicorn `CMD` binding to `0.0.0.0`.
Build and run the API container, then `curl` the `/healthz` readiness endpoint and the `/predict` endpoint to verify it serves once the model is loaded.
What is the difference between a liveness check and a readiness check for a model container?
- Containerize the FastAPI service by running it with `uvicorn app:app --host 0.0.0.0 --port 80` as the `CMD`, with `EXPOSE 80` and an env-driven `MODEL_PATH`.
- `--host 0.0.0.0` is required so requests from outside the container reach uvicorn; binding to localhost would only accept connections from inside.
- A liveness probe ('is the process up?' → restart if not) is distinct from a readiness probe ('is the model loaded?' → withhold traffic until yes); `/healthz` should gate readiness on `model is not None`.
Every cloud-deployed model container ships with a uvicorn entrypoint and a `/healthz` readiness probe — it's how Kubernetes, load balancers, and managed platforms know when to start routing live traffic to a model.
If you remove it: The platform would route traffic to a container before its model finished loading, so the first requests fail — and a slow-loading model could trigger an endless restart loop if liveness and readiness aren't separated.