Intro
Helm is the package manager for Kubernetes, but beyond basic helm install and helm upgrade, there is a set of advanced concepts that make charts production-ready, reusable, and resilient. This article explains those concepts with practical examples, focusing on how they work, when to use them, and how they interact with the Kubernetes API and Helm's release lifecycle.
You will learn about Helm hooks and their role in orchestrating release phases, the Helm lifecycle and how it manages resource ordering, subcharts and global values, chart tests as a verification mechanism, and advanced upgrade strategies like rollbacks and atomic upgrades. For each concept, we provide a concrete command or configuration snippet, explain the expected output, and detail failure signals and recovery steps.
The goal is operational safety: observe before changing, limit the blast radius, use placeholders instead of secrets, verify the result, and document how to recover if the expected state is not reached. Whether you are a developer packaging an application or an operator managing releases, these concepts will help you avoid common pitfalls and build confidence in your Helm workflows.
Version and Environment Inventory
Before diving into advanced concepts, establish a baseline of your Helm environment. This inventory ensures that commands and examples are relevant to your version and cluster setup.
Identify the installed version and deployment topology.
Run the following read-only command from your workstation or CI environment:
helm version --short
Expected output example:
v3.12.2+g4ed1b23
If the command fails with helm: command not found, Helm is not installed or not in your PATH. Verify installation by following the official Helm installation guide for your operating system. Also check that kubectl is configured and can reach your cluster:
kubectl cluster-info
Example output:
Kubernetes control plane is running at https://192.168.1.100:6443
If the connection is refused, ensure your kubeconfig file is valid and the cluster is accessible.
Capture the current observable state.
List all Helm releases in the current namespace with their status and revision:
helm list --namespace <NAMESPACE>
Example output:
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
my-release default 3 2023-09-15 10:20:30.123456 +0000 UTC deployed my-chart-0.1.0 1.16.0
A release stuck in pending-install or failed indicates an issue that must be resolved before proceeding.
Prerequisites and blast radius.
- Helm version 3.10 or higher recommended for full hook support.
- Kubernetes cluster 1.22+ for stable API versions.
- RBAC permissions to create resources in the target namespace.
- A decision owner: for this article, assume the named accountable owner is the Release Manager (e.g., Priya Shah, Engineering Lead). This owner approves any change that affects a shared cluster.
Recovery path.
If environment commands fail, reinstall Helm with the official package manager, verify kubectl connectivity, and ensure the cluster context is correct before any installation attempt.
Safe Configuration Path
Helm charts are configured through values.yaml. Advanced usage involves managing multiple value sources, avoiding secret leakage, and validating changes safely.
Observation before change.
Retrieve the current values of a release:
helm get values my-release --namespace <NAMESPACE>
Example output:
replicaCount: 2
image:
repository: nginx
tag: "1.25"
service:
type: ClusterIP
Smallest justified change.
Instead of editing values.yaml directly, create an override file or use --set for a single scoped change. For example, to change the replica count from 2 to 3:
helm upgrade my-release myrepo/my-chart --namespace <NAMESPACE> --set replicaCount=3 --dry-run
The --dry-run flag simulates the upgrade without modifying the cluster. Expected output includes the generated Kubernetes manifests with replicas: 3. Review the diff before actual upgrade:
helm diff upgrade my-release myrepo/my-chart --namespace <NAMESPACE> --set replicaCount=3
If helm diff is not available, install the helm-diff plugin. This command shows a unified diff of changes.
Blast radius and verification.
A failed upgrade can leave a release in a failed state. To minimize risk, use --atomic to automatically rollback on failure:
helm upgrade my-release myrepo/my-chart --namespace <NAMESPACE> --set replicaCount=3 --atomic
On success, output:
Release "my-release" has been upgraded. Happy Helming!
Verify the rollout status:
kubectl rollout status deployment/my-release-nginx --namespace <NAMESPACE>
Expected output:
deployment "my-release-nginx" successfully rolled out
Never place real credentials in values. Instead, reference Kubernetes Secrets or use Helm's --set-file with placeholder files. Example of a placeholder for a secret value:
# values.secret.yaml for development only
apiKey: "REPLACE_WITH_REAL_KEY"
Verification and Diagnostics
Verification in Helm goes beyond checking pod status. Helm provides built-in tests and hooks for validation.
Helm tests.
Define tests in your chart under templates/tests/. Example test-connection.yaml:
apiVersion: v1
kind: Pod
metadata:
name: {{ include "mychart.fullname" . }}-test-connection
labels:
{{- include "mychart.labels" . | nindent 4 }}
annotations:
"helm.sh/hook": test
spec:
containers:
- name: wget
image: busybox
command: ['wget']
args: ['{{ include "mychart.fullname" . }}:{{ .Values.service.port }}']
restartPolicy: Never
Run the tests after an install or upgrade:
helm test my-release --namespace <NAMESPACE>
Expected output:
NAME: my-release
LAST DEPLOYED: ...
NAMESPACE: default
STATUS: deployed
REVISION: 4
TEST SUITE: my-release-test-connection
Last Started: ...
Last Completed: ...
Phase: Succeeded
If the test fails, the phase will be Failed. Inspect logs:
kubectl logs my-release-test-connection --namespace <NAMESPACE>
Common failure: service name or port misconfiguration. Adjust the test args accordingly.
Diagnosing release failures.
When a release is in failed state, retrieve its status and recent events:
helm status my-release --namespace <NAMESPACE>
kubectl get events --namespace <NAMESPACE> --sort-by=.metadata.creationTimestamp
Example event indicating failure:
Warning FailedCreate replicaset/my-release-6f8c9d7b9 Error creating: pods "my-release-6f8c9d7b9-" is forbidden: unable to validate against any pod security policy
This indicates missing RBAC or PodSecurityPolicy restrictions. Resolve by adjusting permissions or security context.
Failure Modes and Recovery
Advanced Helm operations can fail for many reasons: invalid chart, insufficient permissions, Kubernetes API errors, or hook timeouts. Understanding common failure modes and recovery strategies is critical.
Hook timeout.
Hooks have a default timeout of 5 minutes. If a pre-upgrade hook takes longer, the release fails with context deadline exceeded. To increase timeout per hook:
metadata:
annotations:
"helm.sh/hook": pre-upgrade
"helm.sh/hook-delete-policy": hook-succeeded
"helm.sh/hook-timeout": "10m"
If a hook fails and blocks the release, you may need to manually delete hook resources:
kubectl delete job my-release-hook --namespace <NAMESPACE>
Then retry the upgrade.
Rollback procedure.
To revert to a previous revision:
helm rollback my-release 1 --namespace <NAMESPACE>
Expected output:
Rollback was a success! Happy Helming!
Verify the release revision and status:
helm list --namespace <NAMESPACE>
The revision number increments after rollback. If rollback fails due to missing secret, check helm history my-release --namespace <NAMESPACE> and ensure the revision secret exists.
Atomic rollback.
Using --atomic in upgrade automatically rolls back on failure. However, if the upgrade succeeded but the application is unhealthy, manual rollback may be needed. Always test rollback in a staging environment.
Operations Checklist
This checklist summarizes key steps for safe and successful use of Helm advanced concepts. Each item has an owner and review frequency.
| Step | Owner | Frequency | Verification command |
|---|---|---|---|
| Confirm Helm version and cluster connectivity | Release Manager (Priya Shah) | Before every use | helm version && kubectl cluster-info |
| Review current release values | Release Manager | Before every change | helm get values <release> -n <ns> |
| Test dry-run upgrade | Developer (Alex Chen) | Every chart change | helm upgrade --dry-run --debug |
| Run chart tests after install/upgrade | Developer (Alex Chen) | After every install/upgrade | helm test <release> -n <ns> |
| Validate rollback procedure | Operations Engineer (Sam Lee) | Monthly | helm rollback <release> 0 --dry-run (if supported; otherwise test in staging) |
| Backup release metadata (secrets) | Operations Engineer (Sam Lee) | Weekly | kubectl get secrets -l owner=helm -n <ns> -o yaml > backup.yaml |
| Review and update hook timeouts | Release Manager | Quarterly | Inspect hook annotations in chart templates |
Review frequency is a minimum; adjust based on change rate.
Common Pitfalls and How to Avoid Them
- Ignoring hook delete policies. Hooks that are not cleaned up can clutter the namespace and cause conflicts. Set
helm.sh/hook-delete-policy: hook-succeededfor ephemeral resources like test pods.
- Using
latesttag in images. This makes releases non-reproducible and breaks rollback semantics. Always pin image tags to specific versions.
- Storing secrets in plain values. Even if values are not committed to git, they may be exposed in
helm get values. Use Kubernetes Secrets or external secret management tools with Helm's--setreferencing secret names.
- Forgetting to update dependencies. If using subcharts, run
helm dependency updatebefore packaging to ensure the charts/ directory is up to date.
- Assuming
--waitis enough.--waitwaits for resources to be ready but does not catch custom health conditions. Use Helm tests for application-level verification.
Recovery from these pitfalls usually involves correcting the configuration, deleting stale hook resources, and gradually rolling out changes using --atomic and --timeout flags.
Conclusion
Helm advanced concepts provide powerful mechanisms to manage complex Kubernetes deployments. By understanding hooks, lifecycle management, subchart composition, tests, and safe upgrade practices, you can automate releases with confidence. Always observe before changing, use dry-runs and atomic flags, verify with tests, and have a rollback plan ready.
As a next step, choose one low-risk verification from this article. Run helm version and helm list to observe your current state, then try a --dry-run upgrade on a non-production release. Review the output with your team, and then implement the change with --atomic. Finally, run helm test to confirm the application works as expected. This practice will build a solid foundation for advanced Helm usage.
A reliable technical workflow makes failure visible, protects sensitive values, limits changes to the intended resource, and defines recovery verification before an incident forces the decision. With Helm, that means adopting version control for charts, automated testing in CI/CD, and regular rollback drills.