Health check design: how do I separate Liveness and Readiness?

Short answer: a static {status:ok} only proves the HTTP server is up, not that the app can actually serve. The fix is to separate two different questions: “is it alive?” and “is it ready?”.

The real issue is this: Liveness and Readiness measure different things, and if you mix them up you shoot yourself in the foot — especially if you put the dependency check on the wrong probe.

1. Liveness = “is the process wedged?” — keep it cheap and dependency-free. Don’t check the DB, cache or queue here. If you put a dependency on liveness, a momentary DB blip throws all your healthy pods into an endless restart loop. Liveness should only ask “is the process responding?”.
2. Readiness = “can I serve right now?” — check critical dependencies. Check the dependencies you need to serve here, like the DB, cache and queue. When a dependency goes down, let the Load Balancer pull the pod out of rotation — but without killing it. When the dependency comes back, the pod re-enters traffic.
3. Wire the two outcomes correctly. If liveness fails, Kubernetes restarts the pod; if readiness fails, it just stops traffic. This distinction is critical: on a temporary dependency issue you don’t want to restart the pod, you only want to stop traffic.
4. Keep readiness cheap and cache its result. Do the check with a short timeout and cache the result for a few seconds. Otherwise every replica’s probe runs a real query against Postgres every few seconds; 50 pods × constant probes = you take down your own database. Check only what you truly need to serve.
5. Add a startup probe for slow boots. If your app boots slowly, use a separate startup probe so liveness doesn’t kill the pod while the app is still coming up.

Bottom line: I’d make liveness cheap and dependency-free, and readiness dependency-aware but cached. The golden rule: never let your probes DDoS the database. I separately discuss whether you really need Kubernetes on sade.dev; but if you don’t make this distinction, the smallest DB tremor will shake your whole cluster.