Intro
Kubernetes DaemonSets ensure that a copy of a specific Pod runs on all (or some) nodes in a cluster. They are commonly used for log collection agents, monitoring exporters, storage daemons, and node-level networking components. When a DaemonSet misbehaves, operators need a systematic approach to observe current state, identify deviations, make minimal changes, and verify recovery.
This article provides practical kubectl commands for daily DaemonSet operations. For each scenario, you will see the exact command, a realistic expected output, failure signals, and recovery steps. The focus is on operational safety: always inspect before changing, use read-only commands for diagnosis, and apply changes only when their scope and rollback path are understood.
After reading, you will be able to:
- Confirm the DaemonSet version and environment with
kubectl versionandkubectl cluster-info. - List and describe a DaemonSet and its Pods with labeling and output filtering.
- Check rollout history and status.
- Update a DaemonSet using
kubectl apply,kubectl edit, orkubectl set image, and roll back if needed. - Delete a DaemonSet cleanly, including its Pods.
- Diagnose common failure modes such as unschedulable Pods, crash loops, and node selector mismatches.
- Use a concise cheat sheet for quick reference.
All examples assume kubectl is installed and configured for your cluster. Replace resource names and namespaces with your own values.
Version and Environment Inventory
Before interacting with a DaemonSet, verify that your client and server versions are compatible, and gather basic cluster information.
Commands:
kubectl version --short
kubectl cluster-info
kubectl get nodes
Example output:
Client Version: v1.28.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.2
Kubernetes control plane is running at https://192.168.49.2:8443
CoreDNS is running at https://192.168.49.2:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
NAME STATUS ROLES AGE VERSION
minikube Ready control-plane 10d v1.28.2
Key points to check:
- Client and server minor versions should be within one minor version of each other per Kubernetes version skew policy.
- All nodes should be in
Readystate. If any node isNotReady, DaemonSet Pods on it will not be scheduled or will be evicted. - For managed clusters (EKS, GKE, AKS), node status may be aggregated; use
kubectl get nodes -o widefor details.
Environment discovery for a DaemonSet:
Use kubectl get daemonsets --all-namespaces to see all DaemonSets and their namespaces.
kubectl get daemonsets --all-namespaces
Example output:
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
kube-system kube-proxy 1 1 1 1 1 <none> 10d
logging fluentd-agent 3 3 3 3 3 <none> 5d
monitoring node-exporter 3 3 3 3 3 <none> 5d
If the DaemonSet you expect is missing, check the namespace and the deployment method. For tools installed via Helm, use helm list -A to confirm the release exists.
Listing and Describing DaemonSets
List all DaemonSets in the current namespace
kubectl get daemonsets
Example output:
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
node-exporter 3 3 3 3 3 <none> 5d
List DaemonSets across all namespaces
kubectl get daemonsets --all-namespaces
Describe a specific DaemonSet
Describing a DaemonSet shows its configuration, recent events, and status. This is the first diagnostic step when something goes wrong.
kubectl describe daemonset node-exporter -n monitoring
Key sections in the output:
Selector:labels used to match Pods.Node-Selector:nodes where Pods are scheduled.Tolerations:allow scheduling on tainted nodes.Update Strategy:RollingUpdate or OnDelete.Events:recent scheduling or update events that can reveal problems.
Show DaemonSet labels and selectors
kubectl get daemonset node-exporter -n monitoring -o yaml
kubectl get daemonset node-exporter -n monitoring -o jsonpath='{.spec.selector.matchLabels}'
Example output:
{"app":"node-exporter"}
List Pods owned by a DaemonSet
kubectl get pods -l app=node-exporter -n monitoring -o wide
Example output:
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
node-exporter-abcde 1/1 Running 0 5d 10.244.1.5 worker-1 <none> <none>
node-exporter-fghij 1/1 Running 0 5d 10.244.2.5 worker-2 <none> <none>
node-exporter-klmno 1/1 Running 0 5d 10.244.3.5 worker-3 <none> <none>
Observe whether a Pod exists for every intended node. If a node is missing a Pod, check node taints, labels, and DaemonSet tolerations.
Viewing DaemonSet Configuration and YAML
To understand exactly what a DaemonSet does, inspect its full YAML definition.
kubectl get daemonset node-exporter -n monitoring -o yaml
Save it to a file for editing or backup:
kubectl get daemonset node-exporter -n monitoring -o yaml > node-exporter-ds.yaml
Examine key fields:
spec.template.spec.containers[].image: the container image and tag.spec.updateStrategy:RollingUpdateorOnDelete.spec.minReadySeconds: minimum seconds a Pod must be ready before considered available.spec.revisionHistoryLimit: number of old ReplicaSets to retain for rollback.
Checking Rollout History and Status
DaemonSets support rolling updates, similar to Deployments. To verify that an update completed successfully, use rollout commands.
Check rollout status
kubectl rollout status daemonset/node-exporter -n monitoring
Successful output:
daemonset "node-exporter" successfully rolled out
If the rollout is stuck, the command may time out or print messages like:
Waiting for daemon set "node-exporter" rollout to finish: 2 of 3 updated pods are available...
View rollout history
kubectl rollout history daemonset/node-exporter -n monitoring
Example output:
daemonset.apps/node-exporter
REVISION CHANGE-CAUSE
1 <none>
2 kubectl set image daemonset/node-exporter node-exporter=prom/node-exporter:v1.6.1 --record=true
Note: --record=true is deprecated in newer versions; change-cause is no longer stored automatically.
To see details of a specific revision:
kubectl rollout history daemonset/node-exporter -n monitoring --revision=2
Updating a DaemonSet
Updates can be made via kubectl apply, kubectl edit, or kubectl set image. Always check the update strategy first.
Check update strategy
kubectl get daemonset node-exporter -n monitoring -o jsonpath='{.spec.updateStrategy.type}'
Example output:
RollingUpdate
Apply a manifest change
Edit the YAML file you saved earlier, or apply a new one:
kubectl apply -f node-exporter-ds.yaml
Then monitor the rollout with kubectl rollout status daemonset/node-exporter -n monitoring.
Update container image with kubectl set image
kubectl set image daemonset/node-exporter node-exporter=prom/node-exporter:v1.7.0 -n monitoring
Example output:
daemonset.apps/node-exporter image updated
Verify the new image is used:
kubectl get daemonset node-exporter -n monitoring -o jsonpath='{.spec.template.spec.containers[0].image}'
Edit live object
kubectl edit daemonset node-exporter -n monitoring
This opens the object in your default editor. Changes are applied on save, and a rollout will start if the update strategy is RollingUpdate.
Rolling Back a DaemonSet
If an update introduces problems, roll back to a previous revision.
Rollback to previous revision
kubectl rollout undo daemonset/node-exporter -n monitoring
Example output:
daemonset.apps/node-exporter rolled back
Rollback to a specific revision
kubectl rollout undo daemonset/node-exporter -n monitoring --to-revision=1
After rollback, check status and Pod versions:
kubectl rollout status daemonset/node-exporter -n monitoring
kubectl get pods -l app=node-exporter -n monitoring -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.containers[0].image}{"\n"}{end}'
Deleting a DaemonSet
Deleting a DaemonSet removes the DaemonSet controller and, by default, all Pods it created.
Delete the DaemonSet and its Pods
kubectl delete daemonset node-exporter -n monitoring
Example output:
daemonset.apps "node-exporter" deleted
Delete only the DaemonSet without deleting Pods
Use --cascade=orphan to leave Pods running. This is rarely desired but can be useful during migration.
kubectl delete daemonset node-exporter -n monitoring --cascade=orphan
The orphaned Pods will continue to run, but they will no longer be managed by a DaemonSet. You can adopt them later with a new controller or delete them manually.
Diagnostic Commands for DaemonSet Pods
When Pods are not running correctly, use the following commands to gather information.
Check Pod status
kubectl get pods -l app=node-exporter -n monitoring -o wide
Look for non-Running statuses: Pending, CrashLoopBackOff, ImagePullBackOff, Evicted, etc.
Describe a problematic Pod
kubectl describe pod node-exporter-abcde -n monitoring
Key parts of the output:
Events:show recent scheduling and lifecycle events, such as failed image pulls or insufficient resources.Conditions:indicate Pod readiness and other status.Tolerations:show which taints the Pod tolerates.
View logs
kubectl logs node-exporter-abcde -n monitoring
For a crash-looping Pod, get logs from the previous container instance:
kubectl logs node-exporter-abcde -n monitoring --previous
Execute commands inside the container
kubectl exec -it node-exporter-abcde -n monitoring -- /bin/sh
Then run diagnostic commands like ps aux, df -h, or cat /etc/hostname.
Common Failure Modes and Recovery
Failure 1: DaemonSet Pods not scheduled on some nodes
Symptom: kubectl get pods -l app=node-exporter -o wide shows fewer Pods than nodes.
Possible causes and checks:
- Node selector mismatch:
kubectl get ds node-exporter -o yaml | grep -A5 nodeSelector. - Taints and tolerations:
kubectl describe node <node-name> | grep Taints, and compare with DaemonSettolerations. - Node is cordoned:
kubectl get node <node-name>showsSchedulingDisabled. - Resource insufficient:
kubectl describe node <node-name> | grep -A10 "Allocated resources".
Recovery:
- Adjust node selector or tolerations in the DaemonSet spec and apply.
- Uncordon the node with
kubectl uncordon <node-name>if cordoned. - Free up resources or adjust requests.
Failure 2: Pods in CrashLoopBackOff
Symptom: kubectl get pods shows CrashLoopBackOff or many restarts.
Diagnostic:
kubectl logs <pod-name> -n <namespace> --previous
kubectl describe pod <pod-name> -n <namespace>
Look for exit codes and error messages. Common causes: misconfiguration, missing dependencies, incorrect command arguments.
Recovery:
- Fix the configuration and update the DaemonSet.
- If the image is bad, roll back to a previous version.
Failure 3: ImagePullBackOff
Symptom: Pods stuck in ImagePullBackOff or ErrImagePull.
Diagnostic:
kubectl describe pod <pod-name> -n <namespace> | grep -A5 Events
Check that the image name and tag are correct, and that the node can access the registry (network, credentials).
Recovery:
- Correct the image reference.
- If using a private registry, ensure
imagePullSecretsare set in the DaemonSet spec. - Roll back if necessary.
Failure 4: Rolling update stuck
Symptom: kubectl rollout status hangs, and not all Pods are updated.
Diagnostic:
kubectl describe daemonset <name> -n <namespace>for events.- Check Pod status on individual nodes; often due to unschedulable new Pods (see Failure 1).
- Verify
maxUnavailableandminReadySecondssettings.
Recovery:
- Resolve the underlying scheduling or Pod issue.
- If needed, pause the rollout:
kubectl rollout pause daemonset/<name> -n <namespace>. - Resume with
kubectl rollout resume daemonset/<name> -n <namespace>after fixing.
Operations Checklist
Use this checklist before and after DaemonSet operations to reduce risk.
Before any change:
- [ ] Verify cluster health:
kubectl get nodesall Ready. - [ ] List current DaemonSets:
kubectl get ds -n <namespace>. - [ ] Describe target DaemonSet:
kubectl describe ds <name> -n <namespace>. - [ ] Export current YAML to backup:
kubectl get ds <name> -n <namespace> -o yaml > backup.yaml. - [ ] Check update strategy and rollout history:
kubectl get ds <name> -o jsonpath='{.spec.updateStrategy.type}'andkubectl rollout history ds/<name>. - [ ] Understand node selector and tolerations to ensure Pods will schedule as expected.
After any change:
- [ ] Run
kubectl rollout status ds/<name> -n <namespace>and confirm success. - [ ] Check Pod status across nodes:
kubectl get pods -l <selector> -o wide -n <namespace>. - [ ] View logs of new Pods for runtime errors.
- [ ] Verify application behavior (e.g., metrics endpoint, logs collected).
- [ ] If failure, rollback with
kubectl rollout undo ds/<name> -n <namespace>and verify again.
DaemonSet Commands Cheat Sheet
The following table summarizes essential commands for quick reference.
| Operation | Command |
|---|---|
| List DaemonSets in current namespace | kubectl get daemonsets |
| List DaemonSets in all namespaces | kubectl get daemonsets --all-namespaces |
| Describe a DaemonSet | kubectl describe daemonset <name> -n <namespace> |
| Get YAML of a DaemonSet | kubectl get daemonset <name> -n <namespace> -o yaml |
| Check rollout status | kubectl rollout status daemonset/<name> -n <namespace> |
| View rollout history | kubectl rollout history daemonset/<name> -n <namespace> |
| Update image | kubectl set image daemonset/<name> <container>=<new-image> -n <namespace> |
| Rollback to previous | kubectl rollout undo daemonset/<name> -n <namespace> |
| Delete DaemonSet | kubectl delete daemonset <name> -n <namespace> |
| List Pods belonging to DaemonSet | kubectl get pods -l <selector> -n <namespace> |
| Get logs of a DaemonSet Pod | kubectl logs <pod-name> -n <namespace> |
| Execute command in Pod | kubectl exec -it <pod-name> -n <namespace> -- /bin/sh |
Conclusion
Mastering DaemonSet commands is essential for anyone operating Kubernetes clusters. By following the structured approach in this article, you can confidently inspect, update, and recover DaemonSets while minimizing risk. Always start with read-only commands to observe the current state, make one scoped change at a time, and verify the result with rollout status and Pod checks. Keep backups of YAML definitions and understand rollback procedures. With these practices, you can keep node-level services running smoothly across your cluster.