Senior DevOps / SRE — DevOps interview questions

What is the difference between a liveness and a readiness probe?

A readiness probe decides whether a Pod receives traffic; failing it removes the Pod from the Service endpoints but leaves it running. A liveness probe decides whether the container is restarted. Confusing them is a classic outage: pointing liveness at a dependency means a slow database restarts every replica simultaneously, turning degraded service into total downtime. Liveness should test only "is this process wedged", readiness should test "can I serve right now".

What is Terraform state and why is it dangerous?

State maps the resources declared in your configuration to the real objects in the provider, so Terraform can tell what to create, change or destroy. It is dangerous because it is authoritative: if it is lost, Terraform no longer knows it owns your infrastructure and will try to recreate it. It also frequently contains secrets in plaintext. That is why state belongs in a remote backend with encryption, versioning, and locking so two engineers cannot apply at once.

When do you use Terraform versus Ansible?

Terraform is declarative provisioning: it creates and destroys infrastructure and converges the real world to your declared state. Ansible is procedural configuration management: it runs ordered tasks against machines that already exist. The clean split is Terraform for anything with a lifecycle in a cloud API, Ansible for what happens inside a VM. On a Kubernetes-based stack, Ansible's role shrinks a lot, because container images replace machine configuration.

What are SLI, SLO and an error budget?

An SLI is a measured indicator of service health, such as the fraction of requests served successfully under 300 ms. An SLO is the target for that indicator over a window, for example 99.9% over 30 days. The error budget is the allowed shortfall — 0.1% of requests — and it turns reliability into a spendable resource: if the budget is intact you can ship faster, and if it is exhausted you freeze features and fix reliability. That is what stops "reliability versus velocity" being an argument about opinions.

What is the difference between metrics, logs and traces?

Metrics are cheap numeric aggregates over time and answer "is something wrong" — they are what you alert on. Logs are discrete events with detail and answer "what exactly happened" in one request or component. Traces follow a single request across services and answer "where did the time go" in a distributed call chain. Alerting on logs is expensive and noisy; debugging from metrics alone is guesswork. You need all three, used for what each is good at.

Compare rolling, blue-green and canary deployments.

Rolling replaces instances gradually, needs little extra capacity, but during the roll two versions serve traffic and rollback is slow. Blue-green runs a full second environment and switches traffic at once, giving instant rollback at the cost of double capacity and a hard database-migration problem. Canary sends a small slice of traffic to the new version and watches metrics before widening, which catches issues real users hit but requires solid observability and automated analysis to be worth it.

How should secrets be handled in a pipeline?

Secrets never live in the repository, in image layers, or in plain environment variables baked at build time. They come from a dedicated store — Vault, cloud secret manager, or the CI provider's encrypted secrets — and are injected at runtime with the narrowest scope and shortest lifetime you can manage. Short-lived credentials issued through OIDC federation are better than long-lived keys, because the main risk is not theft of a secret but a secret that stays valid for years.

A Pod is in CrashLoopBackOff. Walk through your diagnosis.

Start with kubectl describe pod to see events — image pull failures, failed mounts, OOMKilled, or a failing probe all show there. Then kubectl logs with --previous, because the current container may be too young to have logged anything useful. Check whether the exit code suggests OOM, in which case compare the memory limit against actual usage. If the container starts and dies immediately, the usual causes are a missing config or secret, a failing migration on startup, or a liveness probe with too short an initial delay.

What makes a postmortem useful rather than ceremonial?

A useful postmortem is blameless, so people describe what they actually did rather than what looks defensible, and it focuses on the conditions that let a mistake become an outage. It records a precise timeline, distinguishes trigger from root cause, and produces a small number of owned, dated action items rather than a wish list. The test is whether the document would let a new engineer understand the failure a year later, and whether the follow-ups actually get done.