Follow the control path from desired state to scheduled Pod to ready endpoint. Good answers distinguish scheduling, process health, traffic readiness, resource enforcement, and application shutdown instead of treating Kubernetes as a restart button.
Developers operating containerized services on Kubernetes · Targets current Kubernetes workload APIs, with version-sensitive features called out. YAML is illustrative and requires validation against the cluster version, admission policy, and application behavior.
What belongs in a Pod, and what should outlive it?
A Pod is the smallest schedulable workload unit. Its containers share a network identity and may share volumes, while the Pod itself can be replaced; durable application data should live in an explicitly managed storage system.
A web app and a local log sidecar can share a Pod when they must share lifecycle or local files. Two unrelated services usually belong in separate Pods so they can scale and recover independently. A Pod's IP and container writable layer are not stable identities.
A volume can persist beyond an individual container restart, but its durability and availability depend on the volume type and storage provider. Stateful applications need identity, recovery, and backup plans beyond merely mounting a path.
YAML · Worked example
apiVersion: v1
kind: Pod
metadata:
name: web
spec:
containers:
- name: app
image: example/web:1.0
- name: log-sidecar
image: example/log-agent:1.0
Illustrates a shared Pod network and lifecycle boundary only. A real workload would normally use a Deployment and would need explicit volume and security configuration.
Illustrative, not executed.
Follow-up: When would a StatefulSet fit better than a Deployment?
When replicas need stable ordinal identity and stable per-replica storage semantics. StatefulSet still does not provide database replication, backups, or application-level recovery by itself.
Use a Deployment for interchangeable long-running replicas, a StatefulSet when stable per-replica identity and storage are needed, and a Job or CronJob for work that completes. Select from lifecycle semantics, not from a generic preference for one controller.
A stateless HTTP service can have several interchangeable Pods behind a Service. A schema export that runs to completion is a Job. A clustered database may require stable identities and persistent claims, but the database still needs its own replication and recovery design.
A Deployment manages ReplicaSets and gradually changes the Pod template. A Job tracks completion and retries according to its policy. Choosing the wrong controller changes how replacement, identity, and completion are represented.
A finite export is modeled as a Job. The retry count is illustrative; a real export needs idempotency or an output strategy that handles retries safely.
Illustrative, not executed.
Follow-up: What makes a retried Job safe?
Make its output idempotent or write to an attempt-specific temporary location and publish only after success. Job retry controls rerun behavior but cannot undo an external side effect already performed.
What does a controller do when actual state drifts?
A controller observes API objects and repeatedly works toward the desired state recorded in them. A Deployment requesting three replicas keeps reconciling its ReplicaSets and Pods toward that count; it does not execute a one-time script and stop.
If an operator deletes one Pod from a three-replica Deployment, the controller creates a replacement. If a node fails, the scheduler and controllers attempt to place the workload elsewhere when constraints and capacity allow. The desired replica count is an intent, not a guarantee that capacity exists.
Because reconciliation is ongoing, manual edits to generated Pods can be overwritten by the controller. Change the owning workload template or the higher-level configuration source. Read object status and events to learn whether the control loop is progressing or blocked.
YAML · Worked example
apiVersion: apps/v1
kind: Deployment
metadata:
name: api
spec:
replicas: 3
selector:
matchLabels: { app: api }
template:
metadata:
labels: { app: api }
spec:
containers:
- name: app
image: example/api:1.0
The Deployment expresses the desired replica count and Pod template. If the image cannot be pulled or nodes lack capacity, reconciliation continues but the requested state is not yet achieved.
Illustrative, not executed.
Follow-up: How do you tell whether reconciliation is stuck?
Inspect the owning controller's status, ReplicaSets, Pod conditions, and recent events. Identify whether the blocker is scheduling, image retrieval, startup, readiness, or policy before changing the desired object.
How does a Service find the Pods that receive traffic?
A Service selects Pods by labels and the control plane publishes matching endpoints. If the selector does not match ready Pods, the Service can exist while having no usable backends.
A Deployment labels Pods app: api, but the Service selector says app: backend. The Service has no endpoints. Check the selector, Pod labels, readiness, and EndpointSlices before debugging application routing code.
Kubernetes Services provide a stable virtual endpoint while matching Pods change. A targetPort must match the container's listening port; containerPort documents the port but does not make the process listen there.
YAML · Worked example
apiVersion: v1
kind: Service
metadata:
name: api
spec:
selector:
app: api
ports:
- port: 80
targetPort: 8080
The selector must match the workload's Pod labels. targetPort 8080 assumes the application actually listens on that port.
Illustrative, not executed.
Follow-up: What would you inspect if the Service has no backends?
Compare the Service selector with Pod labels, inspect EndpointSlices, and check whether readiness is failing. Then test connectivity to the Pod port and verify the app bind address.
How do startup, readiness, and liveness probes differ?
Startup probes gate the other probes while an app initializes; readiness controls whether it receives Service traffic; liveness decides whether kubelet should restart a stuck container. The checks should answer different operational questions.
A process takes two minutes to load a model. A startup probe gives it that startup window. Readiness can remain false while it cannot serve; liveness should detect a genuine unrecoverable stall, not every temporary database outage.
A liveness probe that fails across all Pods under load can cause restart cascades and add work to the remaining replicas. Keep checks cheap, avoid making liveness depend on transient downstream services, and set thresholds from observed startup/recovery behavior.
The numbers and endpoints are illustrative and require measured startup and failure behavior. With a startup probe configured, Kubernetes waits for it to succeed before running liveness and readiness probes.
Illustrative, not executed.
Follow-up: Should readiness fail when a required database is unavailable?
If the app cannot serve useful requests without it, readiness may remove the Pod from traffic while liveness keeps the process alive to recover. Consider whether removing every replica causes a worse failure than returning a controlled degraded response.
Requests inform scheduling and resource weighting; limits constrain runtime usage. CPU limits can throttle a container, while exceeding a memory limit can result in an OOM kill. A request is not a guaranteed ceiling on actual usage.
A container requests 250 millicores and has a 1 CPU limit. The scheduler uses the request when placing the Pod; under contention, a CPU limit can throttle work and raise latency. A memory limit is enforced reactively by the kernel, and a process exceeding it may be killed.
Size values from measured working sets and load, then observe throttling, restarts, and scheduling failures. If requests exceed cluster capacity, Pods remain Pending even when current average utilization appears low.
These values are examples, not recommendations. The memory limit may lead to OOM termination under peak allocation; CPU throttling may affect latency. Measure the application and node capacity before setting values.
Illustrative, not executed.
Follow-up: What does OOMKilled suggest?
The container exceeded a memory limit or was selected by the kernel under memory pressure. Check container and node events, memory usage near the failure, and memory-backed volumes before raising limits blindly.
How can a rolling update stall or reduce availability?
A Deployment creates a new ReplicaSet and scales old and new Pods according to rollout strategy. maxSurge and maxUnavailable bound extra and unavailable replicas, but readiness, capacity, image pulls, and application compatibility can still stall or break a rollout.
A three-replica service uses maxUnavailable: 0 and maxSurge: 1, but the cluster has no room for a fourth Pod. The new Pod stays Pending, so the update cannot progress. If readiness never succeeds, old replicas may remain but rollout completion still stalls.
Database schema changes need compatibility across old and new application versions during overlap. A rollback of the image does not automatically reverse a destructive data migration. Check rollout status, events, Pod conditions, and application logs before changing replica counts.
For small replica counts, integer values make the desired bound clear. Capacity must exist for the surge Pod, and readiness must become true for rollout progress.
Illustrative, not executed.
Follow-up: What do you check before rolling back?
Confirm which ReplicaSet is serving, why the new Pods failed, and whether database or message changes remain compatible. Roll back the workload only after verifying the old version can safely run against the current state.
What should an application do during Pod termination?
Stop taking new work, drain or finish in-flight work within the grace period, close resources, and exit after receiving the termination signal. The application shutdown deadline must fit inside Kubernetes' termination grace period.
During a rollout, Kubernetes begins Pod termination and the EndpointSlice readiness changes. The container process receives a termination signal and is given a grace period; after that, the kubelet can force termination. A preStop hook consumes time from that same budget.
Long-running jobs need durable checkpoints or an external queue. A process that assumes every handler completes can be killed while it is writing. Make side effects idempotent and make cleanup bounded rather than waiting forever.
The grace period is illustrative. The application, preStop hooks, sidecars, and worst-case drain time must be measured together; this field does not implement graceful shutdown by itself.
Illustrative, not executed.
Follow-up: Does a readiness probe have to fail before a Pod is deleted?
No. Deletion updates endpoint conditions as termination proceeds. Readiness remains useful for lifecycle periods when the Pod exists but should not receive normal traffic, such as initialization or temporary overload.
How do you test readiness and rollout behavior before a release?
Validate the manifest against the API server, then exercise startup, readiness loss, termination, and recovery in an isolated environment with the actual application. A dry run checks object acceptance, not whether the service can handle traffic safely.
A readiness endpoint returns success before the server has loaded its routing table. A schema-valid manifest will apply, but a staging traffic test reveals that premature readiness sends requests into an unready handler. Test both cold startup and a temporary dependency failure.
For capacity-sensitive changes, compare request latency, CPU throttling, memory, and restart counts under a representative load before and after. Keep the same request mix and replica count; a successful rollout alone does not prove the workload has enough capacity at peak traffic.
These are example checks, not commands run here. Server dry-run and diff validate the object change; rollout status only waits for controller completion. A staging test must still exercise application readiness and failure recovery.
Illustrative, not executed.
Follow-up: What would make a rollback threshold actionable?
Set it from a measured service objective and a bounded observation window, then ensure the rollout owner can pause or revert safely. Include database and external side-effect compatibility in the rollback plan.
How do you separate Pending from CrashLoopBackOff?
Start with the Pod phase, conditions, events, and container state. Pending usually points to scheduling or image preparation; CrashLoopBackOff means a container repeatedly exits and restart attempts are delayed. Inspect the specific evidence before changing probes or resources.
For Pending, inspect scheduling events for insufficient CPU/memory, taints, affinity, or unavailable volumes. For a crash loop, inspect the current and previous container logs, exit code, image command, mounted configuration, probes, and OOM status.
Use the narrowest read-only diagnostics first: kubectl describe pod, kubectl logs --previous, and workload events. A successful Pod scheduling event does not prove the application is healthy, and a healthy process does not prove the Service selects it.
Shell · Worked example
kubectl describe pod POD
kubectl logs POD -c app --previous
kubectl get events --sort-by=.lastTimestamp
These commands are diagnostic examples only, not commands run against a cluster. Replace placeholders and inspect the namespace and permissions before use.
Illustrative, not executed.
Follow-up: What does a ready Pod with failed requests suggest?
Check Service selectors/endpoints, target port and bind address, network policy, and application-level dependencies. Pod readiness says the configured readiness check passed, not that every request path is correct.
These are Hacker News job posts that mention these technologies, not employee earnings or a measure of what a skill adds to pay. The amounts summarize annualized advertised minimums in approximate USD. Groups can overlap and differ in roles, locations and dates.
Approximate USD using ECB daily reference rates as of 2026-10-09. Original currencies and offer ranges remain on the source listings. These figures cover posts tagged Kubernetes, rather than every job in the broader technology family.