Kubernetes Resource Quota Commands: A Practical Operations Guide

Intro

Resource quotas in Kubernetes are critical for preventing a single namespace from consuming more than its fair share of cluster resources. They give platform teams a hard enforcement point for CPU, memory, storage, and object counts. Without them, a runaway deployment or a misconfigured autoscaler can starve other workloads and destabilize the entire cluster.

This guide focuses on the operational commands you need to create, inspect, modify, and troubleshoot Kubernetes Resource Quotas. It is written for developers, DevOps consultants, and technical startup teams who manage multi-tenant clusters or shared environments. Every section includes concrete kubectl commands, example YAML manifests, expected output, and recovery steps for common failure modes.

The goal is operational safety: observe before changing, limit the blast radius, use placeholders instead of secrets, verify the result, and document how to recover if the expected state is not reached. By the end, you will be able to enforce resource constraints confidently and diagnose quota-related issues in production.

Version and Environment Inventory

Before you run any quota command, confirm the Kubernetes environment version and the objects you are inspecting. Resource Quota behavior has evolved across releases, especially with the introduction of scopes and priority classes. Always capture the current state before making changes.

Check Kubernetes Version

Run the following to identify the cluster version and ensure compatibility with your quota definitions:

kubectl version --short

Expected output looks similar to:

Client Version: v1.29.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.5

If the client and server major versions differ by more than one minor version, upgrade your kubectl to avoid API compatibility problems. Resource Quota is available in all modern Kubernetes versions, but certain fields like scopeSelector and matchScopes require v1.24 or later.

List All Resource Quotas in a Namespace

To see existing quotas in a namespace, use:

kubectl get resourcequota -n <namespace>

For example, in a namespace called team-a, the output might be:

NAME            AGE   REQUEST   LIMIT
compute-quota   3d    cpu: 2/10, memory: 1Gi/4Gi   cpu: 4/10, memory: 2Gi/4Gi
storage-quota   1d    requests.storage: 100Gi/500Gi

The REQUEST column shows current usage and hard limit for request-type resources, while LIMIT shows the same for limit-type resources. This single command gives you a quick health check.

Inspect Quota Details

Use kubectl describe to get the full specification and current status of a quota:

kubectl describe resourcequota compute-quota -n team-a

Expected output excerpt:

Name:         compute-quota
Namespace:    team-a
Resource      Used   Hard
--------      ----   ----
cpu           2      10
memory        1Gi    4Gi
pods          5      20
services      3      10

If the quota does not appear, verify you are in the correct namespace and have appropriate RBAC permissions. The command kubectl auth can-i list resourcequotas -n team-a should return yes.

Compare Quota Usage Across Namespaces

For a multi-tenant overview, run:

kubectl get resourcequota --all-namespaces

This lists all quotas cluster-wide, which is useful for identifying which teams are approaching their limits. You can pipe it through grep or awk to filter by resource type or utilization percentage.

Safe Configuration Path

Changing a Resource Quota can have immediate effects on running workloads. For example, lowering a hard limit below current usage will not terminate existing pods, but it will block new resource creation. Follow this safe path to minimize risk.

Step 1: Export the Current Quota YAML

Before modifying a quota, save a backup:

kubectl get resourcequota compute-quota -n team-a -o yaml > compute-quota-backup.yaml

Inspect the backup file to understand the current spec. This backup is your recovery point if the new configuration causes problems.

Step 2: Apply a Small, Well-Tested Change

Instead of rewriting the entire quota, apply a minimal patch. For example, to increase the hard limit for CPU requests from 10 to 12, create a patch file or use a strategic merge patch:

kubectl patch resourcequota compute-quota -n team-a --type merge -p '{"spec":{"hard":{"cpu":"12"}}}'

Expected output:

resourcequota/compute-quota patched

Always verify the patch with:

kubectl describe resourcequota compute-quota -n team-a

Confirm the Hard column now shows cpu: 12.

Step 3: Use a Dedicated GitOps Path for Production

For production clusters, do not apply raw YAML changes with kubectl patch. Use a Git repository with a pull request flow. The following snippet shows a minimal Resource Quota manifest in Git:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-quota
  namespace: team-a
spec:
  hard:
    requests.cpu: "12"
    requests.memory: 4Gi
    limits.cpu: "20"
    limits.memory: 8Gi
    pods: "30"
    services: "10"

After merging the PR, a tool like Argo CD or Flux can sync the change. This approach provides an audit trail and easy rollback.

Step 4: Verify That the Quota Is Being Enforced

After a quota change, test that new resources are evaluated against the updated limits. For example, create a simple pod in the namespace:

kubectl create deployment test-pod --image=nginx -n team-a

If the quota is exceeded, the pod will not be created, and you will see an error like:

Error from server (Forbidden): pods "test-pod" is forbidden: exceeded quota: compute-quota, requested: cpu=1, used: cpu=12, limited: cpu=12

If you need to delete the test pod, use kubectl delete pod test-pod -n team-a.

Step 5: Monitor Quota Usage Over Time

Set up a watch to observe quota changes as workloads scale:

kubectl get resourcequota compute-quota -n team-a --watch

This updates the resource usage in real time. You can also integrate with Prometheus metrics like kube_resourcequota to build dashboards and alerts.

Verification and Diagnostics

Verification goes beyond checking the quota spec; it ensures that the quota is correctly applied to new workloads and that your namespace is healthy. Use the following commands and techniques.

Verify a Quota Is Active

Query the quota status with kubectl get and look at the REQUEST and LIMIT columns. A quota is active if it appears in the list and has non-zero hard values.

kubectl get resourcequota compute-quota -n team-a

Output:

NAME            AGE   REQUEST                        LIMIT
compute-quota   5d    cpu: 3/12, memory: 1Gi/4Gi    cpu: 5/20, memory: 2Gi/8Gi

If the REQUEST or LIMIT column is empty, the quota may not be tracking those resource types. Check the spec to confirm which resources are defined.

Check Which Resource Is Blocking a Pod

When a pod fails to schedule or create due to quota, the error message often includes the quota name and the exceeded resource. For more detail, describe the ReplicaSet or Deployment that is trying to create the pod:

kubectl describe deployment my-app -n team-a

Look for events like:

Warning  FailedCreate  5m    replicaset-controller  Error creating: pods "my-app-7b9f8b6c5-" is forbidden: exceeded quota: compute-quota, requested: memory=2Gi, used: memory=3.5Gi, limited: memory=4Gi

This tells you exactly which quota and resource is blocking.

Use kubectl describe quota for Full Status

The description output includes usage for all resources and, if scopes are used, the scope selector. Example:

Name:         compute-quota
Namespace:    team-a
Scopes:       NotTerminating
Resource      Used   Hard
--------      ----   ----
cpu           3      12
memory        1Gi    4Gi
pods          7      30

If Scopes is listed, your quota only applies to pods with that scope. Misconfigured scopes are a common reason why a quota seems not to be enforced.

Simulate a Quota-Exceeding Deployment

To test your quota limits, deploy a workload that would exceed them. Create a file stress-pod.yaml:

apiVersion: v1
kind: Pod
metadata:
  name: stress-pod
  namespace: team-a
spec:
  containers:
  - name: stress
    image: polinux/stress
    resources:
      requests:
        memory: "10Gi"
        cpu: "5"
      limits:
        memory: "12Gi"
        cpu: "6"

Apply it and observe the rejection:

kubectl apply -f stress-pod.yaml

Expected error:

Error from server (Forbidden): error when creating "stress-pod.yaml": pods "stress-pod" is forbidden: exceeded quota: compute-quota, requested: memory=10Gi, used: memory=1Gi, limited: memory=4Gi

This confirms the quota is enforced.

Audit Quota Changes

If you need to know when a quota was last modified, check the resource's metadata:

kubectl get resourcequota compute-quota -n team-a -o jsonpath='{.metadata.managedFields[0].time}{"\n"}'

For a full audit trail, enable Kubernetes audit logging and filter for resourcequotas.

Failure Modes and Recovery

Resource Quota issues often manifest as failed pod creations, stuck deployments, or inability to create services. This section covers common failure modes and recovery steps.

Failure: Pod Creation Fails with "Exceeded Quota"

This is the most common failure. The error message will indicate the quota name and the resource that is over limit. To recover, either increase the quota (if justified) or reduce the resource requests of the pod.

Option 1: Increase the quota

kubectl patch resourcequota compute-quota -n team-a --type merge -p '{"spec":{"hard":{"memory":"6Gi"}}}'

Option 2: Reduce the pod's memory request

Edit the deployment and change resources.requests.memory to a value that fits within the current quota. Then apply the change and the pod should create.

Failure: Quota Does Not Seem to Be Enforced

If pods are created despite exceeding the quota, check the following:

• Scopes: A quota with scopes: ["Terminating"] only applies to pods with activeDeadlineSeconds set. A mismatch can cause the quota to be ignored for regular pods.
• Namespace: Ensure the quota is in the same namespace as the pods.
• Resource Names: The quota must specify the same resource type as the pod's resource field. For example, requests.cpu vs cpu.

Use kubectl describe resourcequota <name> -n <ns> to review the spec.

Failure: Cannot Delete a Namespace Because of Quota

Sometimes a namespace gets stuck in Terminating state because a quota blocks finalizer cleanup. To resolve, first check if there are any remaining resources:

kubectl get all -n stuck-ns

Delete any lingering pods or services. If the quota has finalizers, remove them after backing up:

kubectl patch resourcequota <quota-name> -n stuck-ns -p '{"metadata":{"finalizers":[]}}' --type=merge

Then force delete the namespace:

kubectl delete namespace stuck-ns --grace-period=0 --force

Be extremely cautious with force deletion; only do this after confirming no critical data remains.

Failure: Quota Usage Not Updating

If the usage numbers in kubectl get resourcequota do not change after deleting resources, it could be a delay in the controller or a bug. First, try waiting a few minutes. Then check the controller manager logs:

kubectl logs -n kube-system kube-controller-manager-<node> | grep -i quota

If the controller is healthy but usage is stale, you can force an update by restarting the controller (only if you have cluster admin rights).

Recovery: Rollback a Quota Change

If you applied a quota change that caused an outage, roll back using your backup YAML:

kubectl apply -f compute-quota-backup.yaml

If you use GitOps, revert the commit in Git and let the controller sync. Always keep backups for at least 30 days.

Operations Checklist

Use this checklist before and after making Resource Quota changes in production.

Pre-Change Checklist

• Identify the namespace and quota name.
• Record current usage with kubectl describe resourcequota <name> -n <ns> and save output to a file.
• Back up the quota YAML with kubectl get resourcequota <name> -n <ns> -o yaml > backup.yaml.
• Determine the desired new limits based on projected workload growth.
• Check RBAC permissions for the user making the change with kubectl auth can-i update resourcequotas -n <ns>.
• Notify affected teams if the change might block their deployments.

Change Execution Checklist

• Apply the change via GitOps or kubectl apply/kubectl patch.
• Immediately verify with kubectl describe resourcequota <name> -n <ns> and confirm the Hard column.
• Test by creating a small pod (if appropriate) to ensure the quota behaves as expected.
• Monitor kubectl get resourcequota <name> -n <ns> --watch for a few minutes.

Post-Change Validation Checklist

• Check that existing workloads are still running: kubectl get pods -n <ns>
• Check for new quota violation events: kubectl get events -n <ns> | grep quota
• Confirm that the quota usage is within acceptable bounds.
• Update documentation and dashboards with new limits.
• Ensure the backup is stored in a secure location.

Monthly Audit Checklist

• List all quotas across namespaces: kubectl get resourcequota --all-namespaces
• Identify quotas that are too tight (usage > 80% of hard) or too loose (usage < 20% with no growth).
• Review whether resource requests in deployments are properly set.
• Check if any quotas are orphaned or not being used.
• Plan adjustments based on capacity forecasts.

Conclusion

Kubernetes Resource Quotas are a powerful tool for maintaining fairness and stability in shared clusters. By mastering the commands in this guide, you can enforce resource limits proactively, diagnose quota-related failures quickly, and recover from mistakes safely.

As a next step, choose one low-risk verification from this article, such as listing quotas in a test namespace or simulating an over-quota pod. Record the current state, run the documented commands, compare the results with the expected output, and practice the recovery process. Pay attention to related areas like Limit Ranges, namespace boundaries, and pod specifications, as they directly affect quota enforcement.

A reliable operational workflow makes failure visible, protects sensitive values, limits changes to the intended resource, and defines recovery verification before an incident forces the decision. With these practices, you can keep your Kubernetes clusters healthy and your development teams productive.