Kubernetes common errors and fixes with practical examples: practical implementation guide

Intro

Kubernetes will tell you what is wrong, but only if you know where to look. This guide maps common error messages to causes, shows how to diagnose them quickly, and offers safe, minimal fixes. You will learn a repeatable troubleshooting workflow, practical commands, and small YAML changes that resolve issues without risky guesswork.

Who this is for: developers, DevOps consultants, and technical startup teams who run workloads on Kubernetes and need dependable, fast fixes.

Workflow Overview

Use this repeatable flow to cut time to resolution and avoid guesswork:

1. Capture the error and scope

• Record the namespace, workload type (Deployment, StatefulSet, Job), and exact message.
• Snapshot recent events so transient details are not lost.

kubectl get events -n <ns> --sort-by=.lastTimestamp | tail -n 50

1. Inspect the object

• Describe the failing object and check Conditions, warnings, and last transition times.

kubectl describe pod <pod> -n <ns>
kubectl describe deploy <name> -n <ns>

1. Read logs

• Check both current and previous container logs. Previous shows the last crash.

kubectl logs <pod> -n <ns> -c <container>
kubectl logs <pod> -n <ns> -c <container> --previous --tail=200

1. Validate resources and scheduling

• Look for events like Insufficient cpu/memory, taints, or unsatisfiable node selectors.

kubectl get pod <pod> -n <ns> -o wide
kubectl describe pod <pod> -n <ns> | sed -n '/Events:/,$p'

1. Verify network path

• Confirm Endpoints exist and test in-cluster connectivity.

kubectl get svc <svc> -n <ns>
kubectl get endpoints <svc> -n <ns>

1. Prefer safe rollbacks

• If an urgent outage occurs, roll back to last known good while you diagnose.

kubectl rollout undo deploy/<name> -n <ns>

1. Apply the smallest change

• Patch one knob at a time and re-check Events and readiness.

1. Confirm the fix

• Ensure Ready pods are stable for several minutes and relevant Services have Endpoints.

kubectl get pods, svc, endpoints -n <ns> -o wide

Local Pilot Plan

Start small so you get fast wins and predictable results:

• Scope
• Non-production namespace only (for example, ns=ops-pilot).
• 3 to 5 common error types: ImagePullBackOff, CrashLoopBackOff, Pending PVC, Service without Endpoints, RBAC forbidden.

• Setup
• One sample Deployment with a liveness/readiness probe.
• One Service and optional Ingress (for example, Nginx) to test reachability.
• One PersistentVolumeClaim using your default StorageClass (Ceph or other).

• Targets and checks
• Track time to detect and time to first successful request after fix.
• Use kubectl get events and kubectl describe to verify each resolution.

• Guardrails
• Use rollout undo before deep changes.
• Keep patches minimal and reversible.

This narrow, measurable pilot is easy to inspect locally before expanding.

Common Errors and Fixes

Below are high-signal errors you will meet in practice. Each item explains why it happens, how to diagnose it, and a safe fix.

1) CrashLoopBackOff

Why it happens

• The container starts, then exits. Common causes: bad config, missing env vars, failing init containers, or probes killing the pod.

Diagnose

kubectl describe pod <pod> -n <ns>
kubectl logs <pod> -n <ns> --previous --tail=200

Look for exit code, OOMKilled messages, or probe failures in Events.

Fix

• Roll back if this is a new release:

kubectl rollout undo deploy/<name> -n <ns>

• If probes are too strict, relax them temporarily while you fix the app:

# deployment snippet
livenessProbe:
  httpGet:
    path: /healthz
    port: 8080
  initialDelaySeconds: 20
  periodSeconds: 10
  timeoutSeconds: 2
  failureThreshold: 3
readinessProbe:
  httpGet:
    path: /ready
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 5

• Ensure required env and config files exist (ConfigMap/Secret volume paths, file permissions).

2) ImagePullBackOff / ErrImagePull

Why it happens

• Wrong image name or tag, private registry without credentials, or imagePullPolicy conflicts.

Diagnose

kubectl describe pod <pod> -n <ns> | sed -n '/Events:/,$p'

Look for HTTP 403/401 or not found messages.

Fix

• Correct the image reference. Pin a known-good tag:

containers:
- name: api
  image: registry.example.com/team/api: v1.2.3
  imagePullPolicy: IfNotPresent

• Add a registry pull secret and attach it to the ServiceAccount:

kubectl create secret docker-registry regcred \
  --docker-server=registry.example.com \
  --docker-username=<user> \
  --docker-password=<pass> \
  --docker-email=<[email protected]> \
  -n <ns>

kubectl patch serviceaccount default -n <ns> \
  -p '{"imagePullSecrets":[{"name":"regcred"}]}'

3) Pod stuck in Pending (Unschedulable)

Why it happens

• Not enough CPU/memory, node taints, or strict node selectors/affinity rules.

Diagnose

kubectl describe pod <pod> -n <ns>
kubectl get nodes -o wide
kubectl describe node <node> | sed -n '/Taints:/,/Addresses:/p'

Check Events for Insufficient cpu or memory. Review node labels and taints.

Fix

• Right-size requests and limits:

kubectl set resources deploy/<name> -n <ns> \
  --requests=cpu=100m, memory=128Mi \
  --limits=cpu=500m, memory=512Mi

• Remove overly narrow node selectors or add tolerations for required taints.
• Scale the cluster node pool if capacity is genuinely exhausted.

4) PersistentVolumeClaim Pending or MountVolume.SetUp failed

Why it happens

• No default StorageClass, access mode mismatch (ReadWriteOnce vs ReadWriteMany), size not available, or CSI driver issues (for example, Ceph configuration).

Diagnose

kubectl describe pvc <pvc> -n <ns>
kubectl get sc

Check Events for no storage class configured or failed to provision volume.

Fix

• Set or correct the StorageClass:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data
spec:
  accessModes: ["ReadWriteOnce"]
  storageClassName: fast
  resources:
    requests:
      storage: 10Gi

• Mark a StorageClass as default if needed:

kubectl patch storageclass fast \
  -p '{"metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

• For stuck mounts, delete the Pod (not the PVC) and let the controller recreate it after the volume attaches.

5) Service reachable errors (Connection refused/timeout)

Why it happens

• Service selector does not match pod labels, wrong targetPort, pods not Ready, or NetworkPolicy denies traffic. Ingress/Nginx misroutes also surface as 404 or 502.

Diagnose

kubectl get svc <svc> -n <ns> -o yaml | sed -n '1,80p'
kubectl get endpoints <svc> -n <ns>

Endpoints should list pod IPs and ports. If empty, the selector is wrong or no pods are Ready.

Test from inside the cluster:

kubectl run tmp-shell -n <ns> --rm -i --tty \
  --image=busybox:1.36 --restart=Never -- sh
# inside the shell
wget -qO- http://<svc>.<ns>.svc.cluster.local:<port>/healthz || echo fail

Fix

• Align labels and selectors:

# pod labels
metadata:
  labels:
    app: web
# service selector
spec:
  selector:
    app: web

• Match Service port and containerPort/targetPort. Use numeric targetPort to avoid name mismatches.
• If NetworkPolicy blocks traffic, update it to allow the needed namespace and pod selectors.

6) RBAC: Forbidden (user does not have access)

Why it happens

• The user or ServiceAccount lacks permissions for the resource, verb, or namespace.

Diagnose

kubectl auth can-i get pods -n <ns> --as <user>
kubectl auth can-i create deployments.apps -n <ns> --as system: serviceaccount:<ns>:<sa>

Fix

• Bind the correct Role or ClusterRole to the subject:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: app-ops
  namespace: myns
rules:
- apiGroups: ["", "apps"]
  resources: ["pods", "deployments"]
  verbs: ["get", "list", "watch", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: app-ops-binding
  namespace: myns
subjects:
- kind: ServiceAccount
  name: app-sa
  namespace: myns
roleRef:
  kind: Role
  name: app-ops
  apiGroup: rbac.authorization.k8s.io

7) Helm install/upgrade errors

Common messages

• rendered manifests contain a resource that already exists
• UPGRADE FAILED: another operation in progress

Diagnose

helm list -n <ns>
helm status <release> -n <ns>
helm history <release> -n <ns>

If a resource exists from a stray release or manual apply, identify which release owns it via labels/annotations.

Fix

• If the last upgrade broke, roll back:

helm rollback <release> <revision> -n <ns>

• If a resource exists outside Helm control, either adopt it by aligning labels and annotations carefully or remove it after confirming it is safe, then retry the install.
• For another operation in progress, wait for the lock to clear or clean up failed hooks before retrying.

8) Pod stuck Terminating

Why it happens

• Finalizers on attached resources, hung preStop hooks, or volumes that will not unmount.

Diagnose

kubectl describe pod <pod> -n <ns>
kubectl get pod <pod> -n <ns> -o jsonpath='{.metadata.finalizers}'

Fix (progressive)

• Try a normal delete and wait a short period:

kubectl delete pod <pod> -n <ns>

• If a finalizer on the pod itself is stuck and you are certain cleanup is done, remove it:

kubectl patch pod <pod> -n <ns> -p '{"metadata":{"finalizers":[]}}'

• As a last resort only, force delete:

kubectl delete pod <pod> -n <ns> --grace-period=0 --force

Investigate and fix the underlying cause (for example, slow NFS/CSI unmount or long preStop) to prevent recurrence.

9) Node NotReady, scheduling and eviction issues

Why it happens

• Kubelet not healthy, disk pressure, memory pressure, or network/CNI problems.

Diagnose

kubectl get nodes
kubectl describe node <node> | sed -n '/Conditions:/,/Addresses:/p'

Look for DiskPressure or NetworkUnavailable conditions.

Fix

• Free disk space or increase node volume size.
• Cordon and drain before maintenance:

kubectl cordon <node>
kubectl drain <node> --ignore-daemonsets --delete-emptydir-data

• After fix, uncordon:

kubectl uncordon <node>

10) OOMKilled and CPU throttling

Why it happens

• Memory limit too low (container killed by OOM), or CPU requests too low causing throttling under load.

Diagnose

kubectl describe pod <pod> -n <ns> | sed -n '/Containers:/,/Conditions:/p'

Look for Last State: Terminated reason=OOMKilled and Resources section for requests/limits.

Fix

• Set realistic requests and limits based on observed usage:

resources:
  requests:
    cpu: 200m
    memory: 256Mi
  limits:
    cpu: 1
    memory: 512Mi

• If readiness flaps during GC, increase readiness timeout and initial delay.

Quick diagnostic snippets

• Events tail for a namespace:

kubectl get events -n <ns> --sort-by=.lastTimestamp | tail -n 100

• List non-Ready pods with reasons:

kubectl get pods -A --field-selector=status.phase!=Running -o wide

• Show missing Endpoints:

kubectl get svc -A | awk 'NR==1 || $4=="<none>"'

Conclusion

Speedy, safe fixes come from a clear workflow, not guesswork. Start with a small, local pilot that exercises a handful of high-impact errors. For each issue, confirm the exact message, describe the object, read logs, check Events, and change only one thing at a time. Roll back quickly when needed, and validate with readiness and Endpoints. Once your team can resolve these common cases confidently, expand coverage to more services and automate the checks you repeat most often.