Helm Troubleshooting with Practical Examples: A Field Guide

Intro

Helm is the package manager for Kubernetes, but when a release fails, a template renders incorrectly, or a rollback does not behave as expected, the path from symptom to solution is rarely obvious. Troubleshooting Helm effectively means moving beyond guesswork: you need to identify the installed version, understand the deployment topology, capture the right logs, and make changes that are scoped and reversible.

This guide is written for developers, DevOps consultants, and technical startup teams who operate Helm in real clusters. It focuses on practical diagnosis: Helm errors, Helm logs, Helm commands, and Helm recovery. Every section pairs an observed problem with a concrete command, the expected output, a failure signal, and a recovery decision. The examples are deliberately version-scoped and use placeholders instead of real secrets or production identifiers.

The operational principle throughout is safety: observe before changing, limit the blast radius, protect sensitive values, verify the result, and know how to recover before you touch anything.

Version and Environment Inventory

Before you can troubleshoot a Helm problem, you need to know exactly what you are running. Many issues are caused by version mismatches between the Helm client, the Kubernetes cluster, and the chart. Start with a read-only inventory.

Check Helm client version

helm version --short

Expected output:

v3.12.3+g3a31588

If the command fails, Helm is not installed or not in your PATH. Install it from the official documentation and ensure the version matches what your team standardizes on.

Check Kubernetes cluster version

kubectl version --short

Expected output:

Client Version: v1.27.3
Kustomize Version: v5.0.1
Server Version: v1.28.0

A skew of more than one minor version between client and server can cause API incompatibilities in rendered manifests. If you see a message like "Error from server (NotFound): the server could not find the requested resource", your chart may be using an API version that has been removed in the cluster version.

List installed Helm releases

helm list --all-namespaces

Expected output:

NAME        NAMESPACE   REVISION  UPDATED                                 STATUS    CHART              APP VERSION
my-release  default     3         2024-05-12 10:24:31.123456789 +0000 UTC deployed  my-app-1.2.0      1.0.0

Pay attention to the STATUS column. If a release is in a failed or pending state, that is your first clue. If the UPDATE timestamp is old, you may be looking at a stale release.

Capture environment details

helm env

Expected output includes:

HELM_BIN="/usr/local/bin/helm"
HELM_CACHE_HOME="/home/user/.cache/helm"
HELM_CONFIG_HOME="/home/user/.config/helm"
HELM_DATA_HOME="/home/user/.local/share/helm"
HELM_KUBECONTEXT="prod-cluster"
HELM_NAMESPACE="default"

This shows the current kube-context and namespace, which is crucial because Helm commands default to the context in your kubeconfig. A common mistake is running helm commands against the wrong cluster. Always verify the context before making changes:

kubectl config current-context

Expected output:

prod-cluster

If the context is not what you expect, switch it explicitly before proceeding:

kubectl config use-context staging-cluster

Record the topology

For a specific release, see what Kubernetes resources were created:

helm get manifest my-release -n default

This prints the rendered manifest. You can save it to a file for later comparison:

helm get manifest my-release -n default > my-release-manifest-backup.yaml

Before making any change, capture this baseline. It is your recovery point if a modification goes wrong.

Safe Configuration Path

The most dangerous Helm troubleshooting practices are editing live resources directly or overriding values without understanding the blast radius. A safe configuration path separates observation from intervention.

Inspect current values without changing anything

helm get values my-release -n default

Expected output:

USER-SUPPLIED VALUES:
replicaCount: 2
image:
  repository: nginx
  tag: "1.25.0"
service:
  type: ClusterIP

This shows only the values you or a previous operator supplied at install or upgrade time. It does not show defaults from the chart. To see all effective values:

helm get values my-release -n default --all

This includes computed defaults. Compare the two to understand what is explicitly set versus inherited.

Render templates locally before applying

If you are changing values, first preview the rendered manifests with a local dry run. Create a values override file override.yaml:

replicaCount: 3
image:
  tag: "1.26.0"
resources:
  limits:
    cpu: "500m"
    memory: "512Mi"
  requests:
    cpu: "250m"
    memory: "256Mi"

Then render the chart locally:

helm template my-release ./my-chart -f override.yaml > rendered.yaml

Inspect rendered.yaml for correctness. This does not touch the cluster. If the template has an error, Helm will print a diagnostic, such as:

Error: parse error at (my-chart/templates/deployment.yaml:12): unexpected "}" in if

Fix the template before proceeding.

Validate with a dry run upgrade

To test a change against the live cluster without applying it:

helm upgrade my-release ./my-chart -f override.yaml --dry-run --debug

This simulates the upgrade and prints the diff between the current release and the proposed one. Add --debug to see template output and hook execution. The output ends with:

Release "my-release" has been upgraded. Happy Helming!

Even though it says upgraded, nothing has changed. This is a safe way to catch mistakes.

Make the smallest justified change

If the dry run is clean, apply the upgrade:

helm upgrade my-release ./my-chart -f override.yaml --atomic --wait

• --atomic rolls back the release automatically if the upgrade fails.
• --wait waits for resources to become ready before returning.

If the upgrade fails, Helm will automatically roll back to the previous revision, but only if --atomic was specified. Otherwise, you will need to rollback manually (see Failure Modes and Recovery).

Verification and Diagnostics

After any change, verify the result before considering the issue resolved. Helm provides several read-only diagnostic commands.

Check release status

helm status my-release -n default

Expected output includes:

NAME: my-release
LAST DEPLOYED: Mon May 13 15:04:05 2024
NAMESPACE: default
STATUS: deployed
REVISION: 4

If status is not "deployed", investigate further. A status of "pending-install" or "pending-upgrade" means the operation is still in progress or stuck.

Inspect Kubernetes resources created by the release

Use labels to list resources:

kubectl get all -n default -l app.kubernetes.io/instance=my-release

Expected output shows the pod, service, and deployment. If a pod is not running, describe it:

kubectl describe pod my-release-nginx-6d4cf56db6-c9bgk -n default

Look for events like:

Events:
  Type     Reason     Age   From               Message
  ----     ------     ----  ----               -------
  Warning  Failed     10m   kubelet            Failed to pull image "nginx:1.26.0": rpc error: code = NotFound desc = failed to pull and unpack image
  Warning  Unhealthy  5m    kubelet            Readiness probe failed: Get "http://10.0.0.12:80/": dial tcp 10.0.0.12:80: connect: connection refused

This tells you the image tag is missing or the container is not ready.

Check logs

Fetch logs from a specific pod:

kubectl logs my-release-nginx-6d4cf56db6-c9bgk -n default

If the pod crashed, use --previous to see logs from the previous container instance:

kubectl logs my-release-nginx-6d4cf56db6-c9bgk -n default --previous

Use helm get manifest to verify what Helm actually rendered:

helm get manifest my-release -n default | grep -A5 -B5 "image:"

This shows you the exact image specification in the current revision. Compare it with your values.

Validate hooks

Helm hooks are Kubernetes resources that run at specific points in the release lifecycle. View hooks for a release:

kubectl get jobs -n default -l helm.sh/chart=my-app-1.2.0,helm.sh/hook=pre-install

If a hook fails, the release may be stuck. Check hook logs and delete the failed hook job to allow retry, but only after understanding why it failed.

Failure Modes and Recovery

Helm failures usually fall into a few categories: template rendering errors, upgrade failures, rollback issues, and hook timeouts. Knowing the recovery path for each is essential.

Failed upgrade

If helm upgrade fails and you did not use --atomic, the release may be left in a failed state. Check the release status:

helm list -n default

Output might show:

NAME        NAMESPACE   REVISION  UPDATED                                 STATUS    CHART        APP VERSION
my-release  default     4         2024-05-13 15:04:05.123456789 +0000 UTC failed    my-app-1.3.0 1.0.0

To recover, rollback to the last known good revision:

helm rollback my-release 3 -n default

Expected output:

Rollback was a success! Happy Helming!

If you are not sure which revision to roll back to, view history:

helm history my-release -n default

Output shows each revision with status and description.

Rollback not working

Sometimes a rollback itself fails because the previous revision's resources are no longer valid (e.g., they reference a deleted CRD). In that case, you may need to uninstall and reinstall, but first backup all important data. Inspect the previous manifest:

helm get manifest my-release --revision 3 -n default > previous-manifest.yaml

If the manifest is unusable, consider restoring from a known good chart version:

helm upgrade my-release ./my-chart --version 1.2.0 --reset-values --atomic

This resets to the chart's default values for that version, discarding any user-supplied overrides. Use with caution in production.

Hook timeout

Helm hooks have a default timeout of 5 minutes. If a hook takes longer, the release fails with:

Error: timed out waiting for the condition

To increase the timeout for a specific hook, set timeoutSeconds in the hook's annotation:

apiVersion: batch/v1
kind: Job
metadata:
  name: my-migration
  annotations:
    "helm.sh/hook": pre-upgrade
    "helm.sh/hook-delete-policy": hook-succeeded
    "helm.sh/hook-timeout": "600"

Then retry the upgrade. Alternatively, manually run the job and delete the hook after it succeeds, but the annotation approach is cleaner.

Stuck release

If a release is stuck in "pending-install" or "pending-upgrade", it often means the Kubernetes operation did not complete. List resources and look for stuck objects:

kubectl get all -n default -l app.kubernetes.io/instance=my-release

If a pod is stuck in terminating, force delete it after confirming it is safe:

kubectl delete pod my-release-nginx-6d4cf56db6-c9bgk -n default --grace-period=0 --force

Then check if the Helm operation completes. If not, you may need to rollback or uninstall.

Uninstall and cleanup

When all else fails, uninstall the release and reinstall from a clean state:

helm uninstall my-release -n default --keep-history

The --keep-history flag retains the release history so you can inspect or rollback later. Without it, history is deleted. To fully remove, run without the flag.

Operations Checklist

Use this checklist before and after each Helm troubleshooting session. It enforces the safety principles and makes your actions repeatable. Assign each verification to a single owner (e.g., the on-call engineer) and revisit the checklist monthly or after any incident.

Pre-change checklist

• [ ] Confirm cluster context with kubectl config current-context. Expected: the target cluster, not staging or local. Owner: on-call engineer.
• [ ] Record Helm client and server versions with helm version --short and kubectl version --short. Record any mismatch.
• [ ] List current releases with helm list --all-namespaces. Note any failed releases.
• [ ] Capture current values with helm get values <release> -n <namespace> --all. Save to a file.
• [ ] Capture current manifest with helm get manifest <release> -n <namespace>. Save to a file.
• [ ] Identify the exact resource or value to be changed. Define the expected outcome.
• [ ] Determine blast radius: which resources will be modified? Who consumes them?
• [ ] Prepare a rollback plan: know the previous revision number (helm history) and the command to revert.
• [ ] If secrets are involved, ensure you use placeholders or environment variables, never real values in commands or files.

Post-change verification

• [ ] Run helm status <release> -n <namespace>. Confirm STATUS is deployed.
• [ ] Run kubectl get pods -n <namespace> -l app.kubernetes.io/instance=<release>. Confirm all pods are Running and Ready.
• [ ] Check pod logs for errors: kubectl logs <pod> -n <namespace>. Look for stack traces or connectivity issues.
• [ ] Verify application health endpoint returns expected response (e.g., curl http://service-name/health).
• [ ] If hooks are involved, confirm hook jobs completed successfully: kubectl get jobs -n <namespace> -l helm.sh/hook=pre-upgrade,helm.sh/chart=<chart>. Status should be Complete.
• [ ] Document the change, including before/after values and verification results.

Frequency of review

• This checklist is reviewed monthly during the team's operational review.
• After any failed deployment, the checklist is updated to capture the new failure mode.
• The on-call engineer owns the checklist for their shift. The DevOps lead reviews changes quarterly.

Common Pitfalls and How to Avoid Them

Even experienced operators make these mistakes. Knowing them in advance saves hours.

1. Using the wrong kube-context

Why it happens: You have multiple clusters configured in kubeconfig, and Helm uses the current context by default. A command meant for staging accidentally hits production.

How to avoid: Always run kubectl config current-context before a Helm command, or use --kube-context <context> explicitly. For example:

helm upgrade my-release ./my-chart --kube-context prod-cluster

How to recover: If you already made a change to the wrong cluster, immediately note the timestamp and what was changed, then rollback to the previous revision if possible. If the change was an upgrade that failed, the release may still be intact. If it succeeded but is incorrect, perform a rollback.

2. Editing resources directly instead of using Helm

Why it happens: In a hurry, you use kubectl edit deployment to fix a pod spec. This creates a drift between the Helm release and the live cluster. The next helm upgrade may revert or conflict with your manual change.

How to avoid: Always modify the Helm chart or values and run helm upgrade. If you must make a temporary live edit, document it and revert it as soon as possible by reapplying the Helm-managed state.

How to recover: Use helm get manifest to see the Helm-managed state and compare with live. Run helm upgrade with the correct values to realign.

3. Not using --atomic on upgrades

Why it happens: You assume the upgrade will succeed and omit --atomic. If it fails, the release is left in a failed state, and some resources may have been partially updated.

How to avoid: Always use --atomic for production upgrades. It ensures automatic rollback on failure.

How to recover: Manually rollback: helm rollback my-release <last-good-revision> -n default.

4. Hardcoding secrets in values files

Why it happens: For convenience, you put database passwords or API keys in values.yaml. These may get committed to version control or exposed in logs.

How to avoid: Use Kubernetes Secrets, or reference values from environment variables or a secrets manager. Helm has a built-in --set flag but avoid it for secrets. Instead, use a values file that points to a secret key reference.

How to recover: If a secret was exposed, rotate it immediately. Update the Kubernetes Secret and redeploy.

5. Ignoring hook failures

Why it happens: A pre-upgrade migration job fails, but the upgrade continues because it does not block the deployment in some configurations. The application runs with a bad schema.

How to avoid: Always check hook job status before marking an upgrade successful. Use --wait to wait for resources, including hooks, to become ready.

How to recover: Fix the root cause of the hook failure, then rerun the upgrade. If needed, delete the failed hook job to clear its status, but only after fixing the issue.

6. Version mismatch between Helm and chart

Why it happens: You use an older Helm client with a chart that requires Helm 3.7+, or the chart uses APIs not present in your Kubernetes version.

How to avoid: Check the chart's required Helm version in its Chart.yaml (helmdependencies or engine field). Ensure your client and cluster meet the requirements.

How to recover: Upgrade Helm client to a compatible version, or use an older chart version that works with your current setup.

Conclusion

Helm troubleshooting with practical examples is about turning guesswork into a repeatable process. Start with an accurate version and environment inventory, use safe dry runs and value inspections, verify changes with read-only diagnostics, and know the failure modes and their recovery commands. Each step should be scoped, observable, and reversible where the technology permits.

Copying a command without checking prerequisites and expected output is not an operations procedure. A reliable workflow makes failure visible, protects sensitive values, limits changes to the intended resource, and defines recovery verification before an incident forces the decision.

As a next step, choose one low-risk verification from this guide: confirm the cluster context, render a template locally, or run a dry-run upgrade. Record the current state, run the documented check, compare the result with the expected signal, and review dependencies such as Kubernetes, GitLab CI/CD, and Docker integration points. Then document what you learned for your team.