Kubernetes Event Common Errors and Fixes with Practical Examples

Intro

Kubernetes events are one of the first places operators look when something goes wrong, but they are also one of the most misunderstood. An event is not an error by itself; it is a timestamped record of what happened to an object in the cluster, such as a pod being scheduled, a container crashing, or a volume failing to mount. Common errors like CrashLoopBackOff, ImagePullBackOff, FailedScheduling, or FailedMount appear as events, and each one points to a different underlying problem. This article covers the most frequent Kubernetes event errors, how to read them, how to fix them with practical commands, and how to prevent them from recurring.

We focus on real-world operations for developers, DevOps engineers, and startup teams running Kubernetes in production or staging. The goal is to give you a clear path from observing an event to verifying the fix, without dangerous guesswork. Every recommendation includes concrete kubectl commands, expected output, and recovery steps. Before changing anything, capture the current state and understand the blast radius. Then make one scoped change and verify the result. This operational safety pattern runs through the entire article.

Keep in mind that Kubernetes versions and distributions (vanilla, EKS, GKE, AKS, OpenShift) may differ in event wording or defaults. The commands shown here work on Kubernetes 1.24 and later unless noted. Always check your cluster version with kubectl version before applying fixes.

Version and Environment Inventory

Before diagnosing event errors, know exactly what you are working with. Run these commands to establish a baseline:

kubectl version --short
kubectl cluster-info
kubectl get nodes -o wide

Example output (truncated) for a healthy cluster:

Client Version: v1.27.3
K8s Master: https://192.168.49.2:8443
NAME       STATUS   ROLES           AGE   VERSION   INTERNAL-IP
minikube   Ready    control-plane   10d   v1.26.1   192.168.49.2
worker1    Ready    <none>          10d   v1.26.1   192.168.49.3

This tells you the API server address, node versions, and overall health. If a node is NotReady, events on pods scheduled on that node will cascade. Address node issues first.

Next, identify the namespace and resources you care about. Use kubectl get all -n <namespace> to see pods, services, deployments, and more. For a specific pod, get detailed information:

kubectl get pods -n myapp -o wide
NAME                     READY   STATUS             RESTARTS   AGE
web-5db6d9c8f4-abcde     0/1     CrashLoopBackOff   5          10m
web-5db6d9c8f4-fghij     1/1     Running            0          5m

Here one pod is crash looping. To see events for that pod:

kubectl describe pod web-5db6d9c8f4-abcde -n myapp

Look under the Events section at the bottom. It will show timestamps, reason, and message. Example:

Events:
  Type     Reason     Age   From               Message
  ----     ------     ----  ----               -------
  Normal   Scheduled  10m   default-scheduler  Successfully assigned myapp/web-5db6d9c8f4-abcde to worker1
  Normal   Pulled     9m    kubelet            Container image "nginx:latest" already present on machine
  Normal   Created    9m    kubelet            Created container web
  Normal   Started    9m    kubelet            Started container web
  Warning  BackOff    8m    kubelet            Back-off restarting failed container

This shows the container started but then failed, triggering a backoff. To see why it failed, fetch the logs:

kubectl logs web-5db6d9c8f4-abcde -n myapp --previous

The --previous flag is crucial for CrashLoopBackOff because it gets logs from the previous container instance, which contains the error that caused the restart. Without it, you may only see the current (likely failing) start-up logs.

If the pod is pending due to scheduling issues, look at events for FailedScheduling and node conditions. Always record these outputs in your incident notes. Environment inventory is not a one-time step; repeat it after any change to compare states.

Safe Configuration Path

The safest way to fix event errors is to first reproduce them in a controlled environment. Avoid making changes directly to production manifests without testing. Use a local cluster like minikube, kind, or k3s for quick experiments. For example, to test a manifest before applying to production:

kubectl apply --dry-run=client -f deployment.yaml
# or for server-side validation without persisting:
kubectl apply --dry-run=server -f deployment.yaml

If the manifest passes, you can apply it to a staging namespace. Always keep manifests versioned in Git and use a deployment strategy like rolling update to limit blast radius.

When a pod is continuously restarting, inspect the deployment's rollout status:

kubectl rollout status deployment/web -n myapp

If it is stuck, you can pause the rollout:

kubectl rollout pause deployment/web -n myapp

Then make changes and resume or rollback:

kubectl rollout undo deployment/web -n myapp

For configuration errors like missing ConfigMap or Secret, the pod may fail to start. Use kubectl describe to see the event, then check if the resources exist:

kubectl get configmap,secret -n myapp

If missing, create them from the correct files or values. Avoid embedding secrets in manifests; use Kubernetes Secrets and reference them.

After fixing the configuration, verify the pod is healthy:

kubectl get pods -n myapp -w
# Ctrl+C to stop watching

Watch for the status to become Running and ready 1/1 without restarts. A safe configuration path always includes a rollback plan.

Verification and Diagnostics

Once you have applied a fix, you must verify it works as expected. Do not assume; check with commands and logs.

For a pod that is now running, test connectivity from inside the cluster or via port-forward:

kubectl port-forward pod/web-5db6d9c8f4-abcde 8080:80 -n myapp

Then in another terminal:

curl -I http://localhost:8080

Expected output should include HTTP/1.1 200 OK if the container serves HTTP. If not, examine the logs again.

To diagnose container crashes, get the exit code:

kubectl get pods -n myapp -o jsonpath='{.items[0].status.containerStatuses[0].lastState.terminated.exitCode}'

Common exit codes:

• 0: normal exit (but if it exits immediately, maybe the container has no foreground process)
• 1: application error
• 2: misuse of shell builtins
• 126: command invoked cannot execute
• 127: command not found
• 137: SIGKILL (often OOM kill or pod termination)
• 139: SIGSEGV (segmentation fault)
• 143: SIGTERM (graceful termination)

For example, exit code 137 indicates the container was killed, likely due to memory limits. Check the pod's resource usage and limits:

kubectl top pod -n myapp
kubectl describe pod web-5db6d9c8f4-abcde -n myapp | grep -A5 Limits

If the container exceeded memory, increase the limit or reduce memory usage.

Event errors often require checking cluster-level events:

kubectl get events -n myapp --sort-by=.lastTimestamp

You can filter events by field selector, for example, to see only warnings:

kubectl get events -n myapp --field-selector type=Warning

This helps you focus on problems. If you use monitoring tools like Prometheus or Loki, correlate event timestamps with metrics and logs for a complete picture.

Failure Modes and Recovery

Let's dive into specific common event errors and how to recover from them.

CrashLoopBackOff

What it looks like: Pod status CrashLoopBackOff, events show BackOff warning. Why it happens: Container starts and then exits with a non-zero code, repeatedly. Common causes and fixes:

• Application error: Check logs with kubectl logs <pod> --previous. Fix the code or configuration. For example, if a Python app fails because a module is missing, add the dependency to the image or requirements.
• Missing file or config: The container expects a config file but it is not mounted. Ensure ConfigMap or Secret is mounted correctly.
• Liveness probe misconfiguration: If the liveness probe is too strict, the container may be killed even though it is healthy. Adjust initialDelaySeconds, periodSeconds, timeoutSeconds, and failureThreshold.
• Command or arguments wrong: The pod's command or args may be incorrect. Check the image's entrypoint. If the image expects to run as non-root, set securityContext.runAsUser.

Recovery: Fix the root cause, then restart the pod by deleting it (the deployment will recreate it) or rolling out a new version.

ImagePullBackOff

What it looks like: Pod status ImagePullBackOff or ErrImagePull, events show Failed to pull image. Why it happens: The container image cannot be pulled from the registry. Common causes and fixes:

• Image name or tag incorrect: Double-check spelling, registry, and tag. Use docker pull <image> locally to test.
• Private registry credentials missing: Create a Secret with registry credentials and reference it in imagePullSecrets.
• Registry unreachable: Check network connectivity from nodes to registry.
• Rate limiting: Docker Hub rate limits anonymous pulls. Use authenticated pulls or a mirror.

Recovery: Correct the image reference or credentials, then delete the pod to force a new pull, or update the deployment.

FailedScheduling

What it looks like: Pod stuck in Pending, events show FailedScheduling with reasons like 0/3 nodes are available: 3 Insufficient cpu. Why it happens: No node has enough resources, or there are taints/tolerations, or node selectors/affinity rules cannot be satisfied. Common causes and fixes:

• Resource requests too high: Reduce resources.requests or scale up nodes.
• Taints and tolerations: Nodes may have taints; pods need matching tolerations.
• Node selector mismatch: Pod's nodeSelector may not match any node labels.
• Pod anti-affinity: Rules may prevent scheduling.

Recovery: Adjust the pod spec or cluster capacity. Use kubectl describe pod to see the exact reason.

FailedMount

What it looks like: Container cannot start, events show FailedMount or MountVolume.SetUp failed. Why it happens: Volume cannot be mounted, often due to missing PersistentVolume (PV) or PersistentVolumeClaim (PVC), or incorrect secret/configmap name. Common causes and fixes:

• PVC not bound: Check kubectl get pvc -n <ns>. If pending, there may be no matching PV or StorageClass.
• Secret/ConfigMap missing: Create the required resource or fix the reference.
• NFS or cloud volume issues: Check storage backend connectivity and permissions.

Recovery: Fix the underlying storage or reference, then recreate the pod.

Advanced Diagnostics with Events

For deeper troubleshooting, use JSON output and jq to filter events:

kubectl get events -n myapp -o json | jq '.items[] | select(.reason=="FailedScheduling") | .message'

You can also watch events in real time with kubectl get events -w in a separate terminal while you apply changes. This gives immediate feedback.

If events are not providing enough detail, check the kubelet logs on the node where the pod is scheduled:

journalctl -u kubelet -f
# or if kubelet runs as a container:
docker logs kubelet

These logs often contain more detailed error messages, especially for mount issues or container runtime problems.

For events that are old and purged (the default retention is 1 hour), you may need to rely on logs from monitoring systems or increase event TTL in kube-apiserver if needed for forensics.

Common Pitfalls and How to Avoid Them

Many Kubernetes event errors share avoidable pitfalls. Here are the most frequent ones we see in the field.

Ignoring the --previous flag

When a container is crash looping, kubectl logs <pod> shows logs from the current (often just starting) container. The actual error is in the previous container's logs. Always use --previous for CrashLoopBackOff. Without it, you miss the root cause and waste time.

How to avoid: Make kubectl logs <pod> --previous your first command for any pod that restarts.

Assuming events are errors

Not all events are bad. Normal events like Scheduled, Pulled, Created, and Started are expected. Only Warning events indicate problems. Filtering by type helps you focus.

How to avoid: Use kubectl get events --field-selector type=Warning to see only warnings.

Changing too many things at once

In panic, teams often modify multiple configurations simultaneously. If the pod starts working, they do not know which change fixed it; if it fails, they have a mess. This violates the principle of one change at a time.

How to avoid: Make one scoped change, verify, then proceed. Keep a rollback plan.

Not checking resource limits

Many CrashLoopBackOff issues are due to OOMKilled because memory limits are too low. The pod shows exit code 137. Increasing limits or optimizing memory usage is often the fix.

How to avoid: Set reasonable resource requests and limits, monitor usage with kubectl top, and adjust before limits are hit.

Forgetting imagePullSecrets for private registries

When moving from public to private images, pods fail with ImagePullBackOff because credentials are missing. This is a classic configuration oversight.

How to avoid: Always include imagePullSecrets in your pod spec when using private registries. Test pulling the image manually on a node to confirm credentials.

Not using kubectl describe enough

kubectl describe pod shows the event history that kubectl get events might miss or that has been purged. It also shows container states, probes, and mount information all in one place. Many operators jump straight to logs and miss the event clues.

How to avoid: Make kubectl describe pod <name> part of your initial triage, right after kubectl get pods.

Owners and Review Cadence

In a team setting, troubleshooting Kubernetes events should have clear ownership. We recommend assigning a rotation of on-call engineers who are responsible for initial triage. For example, Priya Shah, Engineering Lead, owns the incident response process and reviews event error trends weekly. The on-call engineer investigates each event error, documents the root cause, and updates the runbook. A post-incident review is held for any event error that causes more than 10 minutes of downtime or repeated occurrences. This ensures that fixes are not just band-aids but lead to long-term improvements.

Operations Checklist

Use this checklist when you encounter a Kubernetes event error. It is a compact version of the workflows described above, with concrete commands.

Conclusion

Kubernetes event errors are manageable if you approach them systematically. Start with environment inventory, use kubectl describe and kubectl logs --previous to read the event story, apply one scoped fix at a time, and verify thoroughly. Common errors like CrashLoopBackOff, ImagePullBackOff, FailedScheduling, and FailedMount have well-known causes and fixes described above. Avoid the pitfalls of ignoring previous logs, mixing changes, and neglecting resource limits. Assign clear ownership and review incidents regularly to improve processes.

As a next step, choose one low-risk verification from this article, record the current state of your cluster, run the documented commands, compare the result with the expected signal, and note any discrepancies. Build a habit of checking events before and after every change.

A reliable operational workflow makes failures visible, protects sensitive values, limits changes to the intended resource, and defines recovery verification before an incident forces the decision. With practice, you will turn Kubernetes events from a source of confusion into a powerful diagnostic tool.