Kubernetes Vertical Pod Autoscale: Common Errors and Fixes with Practical Examples

Intro

Kubernetes Vertical Pod Autoscaler (VPA) dynamically adjusts the CPU and memory requests and limits of pods based on actual usage. While powerful, VPA can introduce subtle issues: pods stuck in Pending, unexpected evictions, incorrect recommendations, and configuration conflicts. This guide covers the most common VPA errors you will encounter in production, how to diagnose them with practical commands, and how to apply fixes safely.

We focus on developers, DevOps engineers, and technical startup teams running Kubernetes in production. Every section includes real-world scenarios, kubectl commands, expected outputs, and concrete YAML snippets. We emphasize operational safety: observe first, change one thing at a time, and verify the result before moving on.

By the end of this article, you will be able to:

• Identify whether VPA is installed and healthy in your cluster
• Diagnose common VPA errors using kubectl and logs
• Apply targeted fixes for resource constraints, misconfigurations, and conflicts
• Use VPA with confidence alongside Horizontal Pod Autoscaler (HPA)

Version and Environment Inventory

Before troubleshooting VPA, you must know exactly what is installed and how it is configured. Start by checking the VPA version and deployment status.

Check if VPA is installed

kubectl get pods -n kube-system | grep vpa

Expected output if VPA is running:

vpa-admission-controller-6c7f8b9d4-abcde   1/1     Running   0          2d
vpa-recommender-5f9c8b7d6-ghijk            1/1     Running   0          2d
vpa-updater-7d8e9f0a1-lmnop                1/1     Running   0          2d

If you see no pods, VPA is not installed. Install it following the official documentation for your Kubernetes version.

Verify VPA API version

VPA uses a Custom Resource Definition (CRD). Check available API versions:

kubectl api-versions | grep autoscaling.k8s.io

Expected output (for a recent cluster):

autoscaling.k8s.io/v1

If you see only v1beta2, your VPA version may be older and have different behaviors. Align your troubleshooting steps with the API version.

Check VPA metrics source

VPA relies on metrics-server or Prometheus. Ensure metrics-server is running:

kubectl get deployment metrics-server -n kube-system

If metrics-server is missing or unhealthy, VPA recommendations will be empty or delayed. Install or fix metrics-server first.

Key checklist for environment inventory:

• VPA components running in kube-system namespace
• API version supports your use case
• Metrics source accessible
• Cluster has sufficient capacity for VPA-managed pods

Safe Configuration Path

A common error is creating a VPA object that conflicts with existing pod resource settings or has invalid parameters. Follow this safe configuration path to avoid misconfigurations.

Step 1: Understand VPA modes

VPA supports four update modes:

• Off: only provides recommendations, no changes
• Initial: sets resource requests at pod creation only
• Recreate: evicts and recreates pods to apply new requests/limits (use with caution)
• Auto: dynamically updates resource requests without eviction, but limits are not changed

The most common error is using Auto mode and expecting limits to be adjusted, which does not happen. VPA only updates requests (and limits if minAllowed and maxAllowed are set and mode is Auto with controlledResources including Limits). Be clear about what VPA can and cannot do.

Step 2: Validate your VPA YAML

Here is a minimal VPA definition:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: my-app-vpa
spec:
  targetRef:
    apiVersion: "apps/v1"
    kind: Deployment
    name: my-app
  updatePolicy:
    updateMode: "Auto"
  resourcePolicy:
    containerPolicies:
    - containerName: '*'
      minAllowed:
        cpu: 100m
        memory: 50Mi
      maxAllowed:
        cpu: 1
        memory: 500Mi
      controlledResources: ["cpu", "memory"]

Common mistakes:

• targetRef.name does not match an existing Deployment
• containerName does not match the container in the pod spec
• minAllowed or maxAllowed missing, causing unbounded recommendations
• controlledResources omits a resource you expect to be managed

Step 3: Apply incrementally

Never apply VPA to a large deployment without testing. Start with a single replica and Off mode to observe recommendations before enabling updates.

kubectl apply -f vpa.yaml
kubectl get vpa my-app-vpa -o yaml

Look for status.recommendation to see what VPA suggests.

Step 4: Monitor events

After applying VPA, check events for anomalies:

kubectl describe vpa my-app-vpa

Expected output includes events like:

Events:
  Type    Reason             Age   From                Message
  ----    ------             ----  ----                -------
  Normal  Updated            10m   vpa-updater         Updated pod specs for my-app-1234

If you see warnings about failed updates, investigate the vpa-updater logs.

Verification and Diagnostics

Once VPA is configured, you must verify it is working correctly and diagnose any errors. This section covers common diagnostic commands and interpretation of results.

Check VPA status and recommendations

kubectl get vpa my-app-vpa -o jsonpath='{.status.recommendation.containerRecommendations[0].target}' && echo

Expected output example:

map[cpu:200m memory:100Mi]

If the output is empty, VPA is not generating recommendations. This often indicates metrics-server is not providing data or the pod has no resource requests initially.

Inspect VPA events

kubectl describe vpa my-app-vpa | tail -20

Look for error events such as:

Warning  FailedGetResourceMetric  5m  vpa-recommender  unable to get metrics for resource cpu: no metrics returned from resource metrics API

This error points to a metrics collection issue.

Check vpa-recommender logs

kubectl logs -n kube-system deployment/vpa-recommender --tail=50

Look for lines containing ERROR or WARN. Common errors:

• failed to get metrics
• could not retrieve pod list
• inconsistent labels

Each log line includes a pod name and reason, giving you a precise lead.

Check pod resource requests and limits

After VPA updates a pod, verify the new requests:

kubectl get pod my-app-1234 -o jsonpath='{.spec.containers[0].resources}' && echo

Expected output:

map[limits:map[cpu:1 memory:500Mi] requests:map[cpu:200m memory:100Mi]]

If the requests are unchanged, VPA may not be updating due to mode Off or a policy constraint.

Troubleshooting flow

1. Confirm VPA components are running.
2. Confirm metrics-server is available.
3. Confirm VPA object is targeted correctly.
4. Confirm update mode is not Off.
5. Confirm no conflict with HPA (explained later).

Failure Modes and Recovery

VPA can cause or be involved in several failure modes. This section details the most common errors, their symptoms, root causes, and recovery steps.

Error 1: Pods stuck in Pending state

Symptom: Pods managed by VPA remain in Pending with events like:

Warning  FailedScheduling  10m  default-scheduler  0/3 nodes are available: 3 Insufficient cpu.

Root cause: VPA has increased resource requests beyond the capacity of any node, or maxAllowed is set too high.

Diagnosis:

kubectl describe pod my-app-1234 | grep -A5 Events

Check the requested CPU/memory and node capacity: kubectl get nodes -o custom-columns='NAME:.metadata.name,CPU:.status.allocatable.cpu,MEMORY:.status.allocatable.memory'

Recovery: Adjust maxAllowed to fit your cluster or add more nodes. You can also temporarily set updateMode: "Off" and manually lower requests to bring the pod back.

kubectl patch vpa my-app-vpa --type='json' -p='[{"op": "replace", "path": "/spec/updatePolicy/updateMode", "value": "Off"}]'

Then edit the deployment to reduce requests manually.

Error 2: VPA evicts pods too frequently

Symptom: Pods are constantly being evicted and recreated due to VPA Recreate or Auto mode with aggressive updates.

Root cause: VPA recommendation changes often because of fluctuating usage, and the update mode triggers evictions on every change.

Diagnosis:

kubectl get events --field-selector reason=Evicted -n my-namespace

Check the vpa-updater logs for eviction decisions.

Recovery: Switch to Auto mode if not already, and consider increasing the minReplicas or using Off mode during peak variability. You can also tune the recommender interval (default 1 minute) by adjusting the recommender deployment flags.

Error 3: HPA and VPA conflict

Symptom: HPA scales pods based on CPU, and VPA also adjusts CPU requests, causing thrashing or unexpected scaling behavior.

Root cause: Both HPA and VPA are managing the same resource metric. This is a known anti-pattern.

Diagnosis: Check if both have targetRef to the same deployment: kubectl get hpa,vpa -n my-namespace

Recovery: Do not use HPA on CPU/memory when VPA is active on those resources. If you must use both, set VPA to Off mode or limit HPA to custom metrics only. Alternatively, use HPA on external metrics and VPA on resource requests.

Error 4: VPA recommendations are stale or empty

Symptom: kubectl get vpa shows no recommendations or recommendations that never change.

Root cause: Metrics-server not running, missing RBAC permissions, or VPA targetRef points to a non-existent workload.

Diagnosis:

• Check metrics-server: kubectl top pods -n kube-system
• Check VPA CRD and controller pods: kubectl logs -n kube-system deployment/vpa-recommender --tail=20
• Verify RBAC: kubectl auth can-i get pods.metrics.k8s.io -n kube-system --as system:serviceaccount:kube-system:vpa-recommender

Recovery: Fix metrics-server or RBAC, or correct the targetRef. After fixing, wait a few minutes for new recommendations.

Error 5: VPA does not update limits

Symptom: Pod requests are updated but limits remain unchanged, causing OOM kills.

Root cause: By default, VPA only updates requests. To update limits, you must explicitly set controlledResources to include Limits and provide minAllowed and maxAllowed for limits.

Diagnosis: Inspect VPA configuration and pod limits.

Recovery: Update VPA resource policy as follows:

resourcePolicy:
  containerPolicies:
  - containerName: '*'
    controlledResources: ["cpu", "memory", "limits.cpu", "limits.memory"]
    minAllowed:
      cpu: 100m
      memory: 50Mi
      limits.cpu: 200m
      limits.memory: 100Mi
    maxAllowed:
      cpu: 1
      memory: 500Mi
      limits.cpu: 2
      limits.memory: 1Gi

Then apply and verify.

Operations Checklist

Use this checklist before and after any VPA change to ensure safe operation.

Before enabling VPA on a workload:

• [ ] Confirm VPA components are healthy: kubectl get pods -n kube-system | grep vpa
• [ ] Confirm metrics-server is available: kubectl top pods -n my-namespace
• [ ] Check for existing HPA on the same deployment: kubectl get hpa -n my-namespace
• [ ] Set VPA updateMode to Off first to observe recommendations without changes
• [ ] Define minAllowed and maxAllowed to prevent out-of-range requests
• [ ] Test on a low-impact deployment with one replica

After VPA is active:

• [ ] Monitor pod restarts: kubectl get pods -w
• [ ] Check for evictions: kubectl get events --field-selector reason=Evicted -n my-namespace
• [ ] Verify actual resource usage: kubectl top pods -n my-namespace
• [ ] Review VPA recommendations: kubectl get vpa -o yaml
• [ ] Ensure no node exhaustion: kubectl describe nodes | grep -A5 'Allocated resources'

Rollback plan:

• Delete the VPA object: kubectl delete vpa my-app-vpa
• Manually set resource requests and limits in the deployment spec
• Redeploy and verify pods are running

Conclusion

Kubernetes Vertical Pod Autoscaler can significantly improve resource utilization and application stability, but it must be configured and monitored carefully. This guide covered common errors such as pods stuck pending, frequent evictions, HPA conflicts, empty recommendations, and unchanged limits. For each, we provided practical commands, YAML examples, and recovery steps.

Remember the core operational principles:

• Observe before changing: always check current state and VPA logs
• Limit blast radius: test with Off mode and small workloads
• Verify after changes: confirm recommendations, pod resources, and cluster capacity
• Have a rollback plan: know how to disable VPA and manually set resources

As a next step, run through the Operations Checklist on one of your deployments. Start with Off mode, watch the recommendations for a day, then gradually switch to Auto with appropriate min/max bounds. Document your findings and share them with your team.

With careful implementation, VPA will help you run a more efficient and resilient Kubernetes cluster.