15 · Deploying on AWS
The image is in ECR — now run it. AWS offers a managed-vs-control spectrum: Elastic Beanstalk (managed, hand it a container), raw EC2 (a VM you wire up yourself), and SageMaker endpoints (fully managed model serving with autoscaling). Picking the right rung is the deploy decision.
AWS deployment options sit on a managed-vs-control spectrum. **Elastic Beanstalk** is managed PaaS: hand it your Docker image (a `Dockerrun.aws.json`) and it provisions the servers, load balancer, and health checks for you. **EC2** is a raw virtual machine: maximum control, but you install Docker, pull the image, run the container, and own the OS, scaling, and uptime yourself. **SageMaker endpoints** are fully-managed *model serving*: deploy a model and SageMaker gives you an autoscaling HTTPS endpoint with built-in monitoring. Pick by how much operational burden you want to trade for control: Beanstalk for a quick managed container, EC2 for full control, SageMaker for managed ML-native serving.
Without this:
Without choosing deliberately, you either over-engineer (hand-managing an EC2 fleet when Beanstalk would have done it for you) or under-provision (a single EC2 box with no autoscaling or health checks behind a service that needed them) — and pay for it in ops time or outages.
Your image is pushed to ECR (lesson 14). Now something has to run it and expose it to the world. AWS gives you several ways, and they line up on one axis: how much do you manage vs how much does AWS manage? More managed = less ops work but less control; more raw = full control but you own everything. Three rungs cover most ML services.
Elastic Beanstalk — managed PaaS, "here's my container, run it". Beanstalk is the easy button. You give it your app — for a containerized service, a tiny Dockerrun.aws.json that points at your ECR image — and Beanstalk provisions everything underneath: EC2 instances, a load balancer, health monitoring, rolling deploys, and autoscaling, all configured for you. You think about the app; AWS thinks about the servers. The trade is less control over the exact infrastructure. Beanstalk is the sweet spot for "I have a Docker image of my FastAPI model service and I just want it running and load-balanced".
EC2 — a raw virtual machine, maximum control. EC2 is a bare VM in the cloud. You SSH in, install Docker, docker pull your image from ECR, docker run it, open the security-group ports, and — crucially — you own the OS patches, the process supervisor that restarts it if it dies, scaling, and uptime. Total control, total responsibility. Pick EC2 when you need something Beanstalk won't give you: a custom OS setup, GPU drivers, unusual networking, or a long-lived job that isn't a simple web service.
SageMaker endpoints — fully-managed, ML-native serving. SageMaker is AWS's machine-learning platform, and a real-time endpoint is its managed model-serving product. You give it a model (and a container that knows how to serve it) and SageMaker stands up an autoscaling HTTPS endpoint with health checks, multi-AZ redundancy, and built-in hooks for monitoring and A/B traffic splitting — purpose-built for serving predictions. It's the most managed and the most ML-aware, at a price premium. Pick SageMaker when model serving is the product and you want autoscaling + monitoring without building it.
Choosing. Default to Beanstalk for a containerized model API you just want running with a load balancer and minimal ops. Reach for EC2 when you need control Beanstalk can't give. Choose a SageMaker endpoint when you want fully-managed, autoscaling ML serving with monitoring built in. The question is always the same: how much operational burden do you want to trade away for control?
AWS deploy paths: Beanstalk (`eb init/create/deploy` — managed), raw EC2 (SSH in, install Docker, pull from ECR, `docker run` — you own ops), and a SageMaker endpoint sketch (managed, autoscaling ML serving).
A Beanstalk `Dockerrun.aws.json` v1: it points Elastic Beanstalk at the exact ECR image (by SHA tag) and maps the container's port 8000 to host port 80 — Beanstalk handles the servers, load balancer, and scaling around it.
Your team has a containerized FastAPI model service in ECR and wants it running, load-balanced, and autoscaled with as little server management as possible — but you do NOT need ML-specific serving features like A/B traffic splitting. Which AWS option fits best?
- AWS deploy options form a managed-vs-control spectrum: Elastic Beanstalk (managed PaaS — hand it a container), raw EC2 (a VM you wire up and own), and SageMaker endpoints (fully-managed ML serving).
- Beanstalk takes a tiny `Dockerrun.aws.json` pointing at your ECR image and provisions the servers, load balancer, health checks, and autoscaling — the easy button for a containerized model API.
- Pick by trade-off: Beanstalk for a quick managed container, EC2 when you need full control AWS won't give, a SageMaker endpoint for autoscaling ML-native serving with monitoring built in.
Production teams run their model API on whichever AWS rung matches the need — Beanstalk or ECS for a managed container, EC2 for special infra, a SageMaker endpoint when autoscaling ML serving is the product — all pulling the image from ECR.
If you remove it: Without choosing deliberately you over-engineer an EC2 fleet Beanstalk could have managed, or under-provision a lone box with no autoscaling or health checks — paying in ops time or outages.