Support says roughly one in five requests to the customer portal is returning a 502. Nothing has been deployed for two days. The pods all look Running and Ready. Walk me through how you find it.
What they are really testing: They are testing whether you can debug a path rather than a component, and whether you understand that a 502 is the ingress telling you the backend failed it, not the backend telling you it is broken.
A real interview question
Support says roughly one in five requests to the customer portal is returning a 502. Nothing has been deployed for two days. The pods all look Running and Ready. Walk me through how you find it.
What most people say
drag me
“A 502 is a backend problem, so I would restart the pods and see if the errors clear up. If they come back I would scale up the deployment.”
It reads junior because a rolling restart hides the evidence without explaining anything, it will appear to work whenever the cause is a single sick replica, and it leaves the real cause, usually a timeout or keepalive mismatch, alive to come back next week.
The follow-ups they ask next
Every pod shows 502s at the same low rate, not just one. What is your new hypothesis?
Move away from a sick replica and toward the connection lifecycle. Keepalive and idle timeout mismatch, no graceful shutdown handling on SIGTERM, or an upstream dependency timing out just under the proxy timeout.
How is readiness passing if the pod is broken?
Because the readiness probe is probably a static /healthz that returns 200 without touching the database or the thread pool. Argue for a readiness check that reflects real dependency health, and a liveness check that does not.
How do you know your fix worked and it did not just move?
Do not trust a five minute quiet window. Watch the 502 rate and the p99 for at least a full traffic cycle, and keep the alert threshold tight until you have seen a peak hour pass clean.
What the interviewer is listening for
- Explains that a 502 is the proxy's report on the backend, not the backend's own error
- Uses the one in five ratio to infer a single bad replica out of five
- Scales out to reduce user pain before starting root cause work
- Names the keepalive versus idle timeout mismatch as a real candidate
What sinks the answer
- Rolling restarts the deployment and calls it fixed
- Never opens the ingress controller logs
- Trusts Ready as proof that the pod is healthy
- Declares victory after two quiet minutes with no traffic cycle observed
If you genuinely do not know
Say this instead of freezing. Reasoning out loud from what you do know beats silence every single time, and a good interviewer is listening for exactly that.
“I have not debugged this exact portal, but here is how I would reason about a 502. It is the proxy saying the backend failed it, so I would first see whether one replica owns most of the errors, take the pain off users by scaling out, and then read the ingress logs to decide between a sick pod and a timeout mismatch.”
Keep going with troubleshooting
Junior
You get paged at 2am. Checkout latency has gone from 200ms to 4 seconds and it started right after the 11pm deploy. Walk me through what you do.
Junior
It is 3am and you get paged: the primary application server is at 100% disk usage and the app is throwing write errors. You are the only one online. Walk me through exactly what you do.
Junior
You push a new version of a service and within two minutes the pods are all showing CrashLoopBackOff. The team is watching you share your screen. Walk me through exactly what you do.
Junior
A developer pings you: the checkout service cannot reach the payments service, the calls just hang and time out. You have never worked on either service. Walk me through what you do.
Junior
A developer messages you: 'My app suddenly gets Access Denied writing to the S3 bucket. Nothing changed.' It worked yesterday. Walk me through what you do.
Mid
Your load balancer shows 5xx errors jumping from 0.1 percent to 12 percent of requests over ten minutes. There was no deploy. What do you do?
Knowing the answer is not the same as recalling it under pressure
Sign in to send the questions you fumble to spaced recall, so they come back right before you would forget them, and learn the concepts behind them with hands-on labs.
Start free