The DevOps interview question that predicts an outage

There is a specific way to turn a slow database into a total outage, and it fits in one line of YAML.

You point your Kubernetes liveness probe at an endpoint that checks the database. The database gets slow. The probe times out. Kubernetes concludes the container is wedged and restarts it. It restarts every replica, all at once, because they all failed for the same reason at the same time. What was a degraded service — slow, but serving — is now nothing at all, and the restarts keep the database busy enough that it cannot recover.

I have never met an engineer who did this on purpose. It happens because liveness and readiness probes look interchangeable in the documentation, and the difference only becomes obvious at three in the morning.

That is exactly why it is a good interview question.

Naming tools is not the same as knowing what to do with them

Most DevOps interviews are a vocabulary test. Do you know what Terraform is. Have you used Kubernetes. Can you list the stages of a pipeline. A candidate who has read the docs passes. A candidate who has run things in production passes identically, and you learn nothing about the difference between them.

The job is not remembering what a Deployment is. The job is deciding what to roll back first when two changes went out together and you can only undo one. It is deciding what deserves to wake someone up. It is deciding what to leave broken until morning — which is a real decision that real engineers make, and one that never appears in a tutorial.

So the useful questions are the ones with a fork in them, where a confident answer either way is fine and the reasoning is the whole point.

Five questions that actually separate people

"What is the difference between a liveness and a readiness probe?"

Everyone gets the definitions. Readiness controls whether traffic arrives; liveness controls whether the container is restarted. The follow-up is where it gets interesting: what happens if your liveness probe checks a dependency?

What you are listening for is whether the candidate sees the amplification. A readiness failure removes one pod from rotation and the rest carry on. A liveness failure kills the pod. Point liveness at something shared and you have built a mechanism that converts partial degradation into simultaneous, cluster-wide restarts. An engineer who has been burned by this answers immediately and with feeling.

"A pod is in CrashLoopBackOff. Give me your first three commands."

This one is unfakeable. There is no clever answer, only a practised one.

A good answer starts with kubectl describe pod, because the events section holds image pull failures, failed mounts, OOMKilled, and probe failures — most of the answer, before you have read a single log line. Then kubectl logs --previous, because the container currently running may be seconds old and have logged nothing useful. Then a look at the exit code, and if it smells like OOM, a comparison of the memory limit against actual usage.

What is telling is not the exact commands but the order. People who debug from experience narrow the space before they start reading. People who have only read about it start with logs and get lost.

"What is an error budget, and how would you use it in an argument with a product manager?"

The definitions part is easy: an SLI is what you measure, an SLO is the target over a window, and the error budget is the gap you are allowed to spend.

The second half of the question is the real one. An error budget is not a metric, it is a negotiating device. It converts "we should slow down and fix reliability" from an opinion — which loses to a roadmap every time — into arithmetic that both sides agreed to in advance. Candidates who have used one describe it that way. Candidates who have only read the SRE book describe the maths and stop.

"What is Terraform state, and why is it dangerous?"

The trap here is answering only the first half. State maps your configuration to real resources so Terraform knows what to create, change, or destroy.

Why it is dangerous is the part that matters. It is authoritative: lose it and Terraform no longer knows it owns your infrastructure, so it offers to build everything again. It frequently contains secrets in plaintext. And two engineers running apply at the same time without locking can corrupt it. A candidate who has actually operated Terraform gets to remote backends, encryption, versioning and state locking without being prompted, because they have felt the fear.

"You are deploying a service with a database migration. Rolling, blue-green, or canary?"

There is no correct answer, which is the point.

Rolling needs almost no spare capacity, but two versions serve traffic during the roll — so the migration has to be backward compatible whether you like it or not. Blue-green gives instant rollback and doubles your infrastructure, and the migration problem gets worse rather than better, because the two environments share one database. Canary catches problems on real users but only pays off if your observability is good enough to notice within the canary window.

Anyone who picks one instantly without mentioning the migration has not thought about it. Anyone who says "it depends" and stops has not either. The answer you want names a choice and then names what it costs.

If you are the one preparing

The pattern across all five is the same: the question sounds like it is about a tool, and it is actually about a decision.

So when you prepare, do not stop at the definition. For every concept you revise, ask yourself what it looks like when it goes wrong, what you would check first, and what you would give up to fix it. If you can only recite what something is, you will answer the first half of the question and go quiet on the second — and the second half is the one being graded.

The other thing worth saying: reading answers is not the same as saying them. Written down, your reasoning looks complete. Out loud, under mild pressure, with someone waiting, it comes out in a different order and half of it goes missing. That gap is not a knowledge problem, and revising harder does not close it. Practice does.

We keep a set of DevOps and SRE interview questions with model answers if you want somewhere to start, and Cam — our AI interviewer — will ask you the same kind of question out loud and tell you where your answer fell apart. Three interviews a month are free.

Good luck. And check where your liveness probe is pointing.