Azure 67: Capstone — Chaos Engineering & a Resilience Scorecard
easy⏱ 5 mincourseazure
Break it on purpose
Chaos engineering injects controlled failure — kill a pod, add 300ms latency, fail a region — and verifies the system survives. Azure Chaos Studio runs these as experiments against real resources (non-prod first!). The goal isn't chaos; it's evidence that your probes, retries and failover actually work before a real incident tests them.
// Chaos Studio fault example
// type: pod-failure, duration: 5m, selector: app=orders
The resilience checklist
A production service should have: health probes (liveness + readiness), an SLO with alerting, distributed tracing, structured logs with a correlation id, timeouts + retries + a circuit breaker on every dependency, and multi-region failover for the critical path. Missing any one is a known way to turn a blip into an outage.
Score the service
Write score(config) over booleans { probes, slo, tracing, structuredLogs, circuitBreaker, multiRegion }. Each true is worth 1 (max 6). Print score: N/6 and a verdict: >=5 → RESILIENT, 3-4 → FRAGILE, <3 → AT RISK.
Start small, in non-prod
Run your first chaos experiment in staging with a tight blast radius and an abort button. Graduate to prod game days only once you trust the guardrails — chaos without a hypothesis and a stop condition is just an outage you caused.
Quiz: what's the point of chaos engineering?
To gather evidence that resilience mechanisms work — controlled experiments surface weaknesses before a real incident does.