Kubernetes Multi-Tenancy: Basic Commands with Practical Examples

Intro

Kubernetes multi-tenancy lets multiple teams, applications, or customers share a single cluster while staying isolated from each other. Getting the basics right requires more than copying commands. You need to know what each command does, what output to expect, and what to do when things go wrong.

This article is for developers, DevOps consultants, and technical startup teams who need practical command-line skills for managing multi-tenant clusters. It covers essential commands with real examples, expected outputs, failure signals, and recovery steps. You will learn how to inspect your environment, apply safe configuration changes, verify tenant isolation, and diagnose common problems.

The goal is operational safety: observe before changing, limit the blast radius, use placeholders instead of secrets, verify the result, and know how to recover if the expected state is not reached.

Version and Environment Inventory

Before running any multi-tenancy command, know what you are working with. Check your Kubernetes version, the current context, and the namespaces that already exist. This helps you avoid running commands against the wrong cluster or accidentally impacting other tenants.

Check cluster version and API availability

Run:

kubectl version --short

Expected output looks like:

Client Version: v1.27.3
Kustomize Version: v5.0.1
Server Version: v1.27.3

The server version tells you which API resources are available. Multi-tenancy features like ResourceQuota and NetworkPolicy have been stable since early versions, but newer features like hierarchical namespaces (HNC) require v1.20+ and are not enabled by default.

Verify current context

Always confirm which cluster and namespace you are operating in:

kubectl config current-context

Expected output example:

gke_my-project_us-central1-a_production-cluster

If it shows a development or test cluster when you expect production, stop. Switching to the wrong context is a common cause of accidental tenant impact.

List existing namespaces

Multi-tenancy often starts with namespace-level isolation. See what namespaces already exist:

kubectl get namespaces

Expected output example:

NAME              STATUS   AGE
default           Active   210d
kube-system       Active   210d
kube-public       Active   210d
team-a            Active   45d
team-b            Active   45d

Each tenant should have at least one dedicated namespace. Avoid placing tenant workloads in default or kube-system.

Inspect current pod and service distribution

See a cluster-wide view of pods:

kubectl get pods --all-namespaces -o wide

This shows which pods belong to which namespace and node. It helps you spot tenants that are co-located or consuming unusual resources.

For a deeper look at a specific pod:

kubectl describe pod <pod-name> -n <namespace>

Look for Node-Selectors, Tolerations, and Resource Requests/Limits. In a multi-tenant environment, these should be set to prevent noisy neighbor problems.

Check resource quotas

Resource quotas are critical for fair sharing. List them across all namespaces:

kubectl get resourcequota --all-namespaces

Expected output example:

NAMESPACE   NAME          AGE   REQUEST                                        LIMIT
team-a      team-a-quota  10d   requests.cpu: 0/4, requests.memory: 0/8Gi     limits.cpu: 0/8, limits.memory: 0/16Gi
team-b      team-b-quota  10d   requests.cpu: 0/2, requests.memory: 0/4Gi     limits.cpu: 0/4, limits.memory: 0/8Gi

If no quotas exist, tenants can consume unlimited resources and starve others.

Inspect RBAC bindings

Role-Based Access Control (RBAC) enforces who can do what inside each tenant namespace. List role bindings for a namespace:

kubectl get rolebindings -n team-a

Expected output:

NAME              ROLE                AGE
team-a-admins     ClusterRole/admin   30d
team-a-devs       Role/developer      30d

This tells you which users or groups have access. Too many bindings to highly privileged roles is a security risk.

Safe Configuration Path

Making changes in a shared cluster requires care. Follow a path that minimizes risk: define the desired state in a manifest, validate it, apply it to a test namespace, and verify the result before rolling out to production tenants.

Use YAML manifests, never imperative one-offs

Imperative commands like kubectl create namespace team-c are fast but not reproducible. Use declarative manifests stored in version control.

Example namespace manifest team-c-namespace.yaml:

apiVersion: v1
kind: Namespace
metadata:
  name: team-c
  labels:
    tenant: team-c
    environment: production

Apply it:

kubectl apply -f team-c-namespace.yaml

Expected output:

namespace/team-c created

Now kubectl get namespace team-c should show Active.

Set resource quotas for a tenant

Create a ResourceQuota to limit CPU and memory for pods in team-c:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: team-c-quota
  namespace: team-c
spec:
  hard:
    requests.cpu: "4"
    requests.memory: 8Gi
    limits.cpu: "8"
    limits.memory: 16Gi
    pods: "20"

Apply and verify:

kubectl apply -f team-c-quota.yaml
kubectl get resourcequota -n team-c

Expected output:

NAME           AGE   REQUEST                                        LIMIT
team-c-quota   5s    requests.cpu: 0/4, requests.memory: 0/8Gi     limits.cpu: 0/8, limits.memory: 0/16Gi, pods: 0/20

Restrict access with RBAC

Create a Role that allows read-only access to pods in team-c:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: team-c
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list", "watch"]

Bind it to a development team group:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: team-c-devs-read-pods
  namespace: team-c
subjects:
- kind: Group
  name: team-c-devs
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

Apply both:

kubectl apply -f role.yaml -f rolebinding.yaml

Verify:

kubectl auth can-i get pods -n team-c --as system:serviceaccount:default:test-user

Expected output if allowed:

yes

Isolate network traffic with NetworkPolicy

By default, all pods in a cluster can talk to each other. In multi-tenant environments, you should deny all traffic between namespaces unless explicitly allowed.

Example default-deny for team-c:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: team-c
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

This blocks all ingress and egress traffic for pods in team-c. You then add specific allow rules.

Test in a staging namespace first

Never apply multi-tenancy changes directly to production namespaces. Use a copy of the production namespace with a suffix like -staging. After applying and verifying in staging, roll forward.

Example flow:

kubectl create namespace team-c-staging
kubectl apply -f team-c-namespace.yaml -n team-c-staging
kubectl apply -f team-c-quota.yaml -n team-c-staging
kubectl apply -f role.yaml -n team-c-staging
kubectl apply -f rolebinding.yaml -n team-c-staging
kubectl apply -f networkpolicy.yaml -n team-c-staging

Then run tests to confirm isolation works as expected.

Verification and Diagnostics

After applying configuration, verify that each isolation mechanism works. Do not assume that because a manifest applied without errors, the policy is enforced correctly. Actively test boundaries.

Verify namespace isolation

Check that pods in one namespace cannot be listed from another namespace using a service account that lacks cross-namespace permissions.

Create a service account in team-a and try to list pods in team-b:

kubectl create serviceaccount sa-test -n team-a
kubectl auth can-i list pods -n team-b --as system:serviceaccount:team-a:sa-test

Expected output:

no

If yes, RBAC permissions are too broad.

Test resource quota enforcement

Try to create a pod in team-c that exceeds the memory quota. Example pod with 20Gi memory request when the quota limit is 16Gi:

apiVersion: v1
kind: Pod
metadata:
  name: test-memory-exceed
  namespace: team-c
spec:
  containers:
  - name: nginx
    image: nginx
    resources:
      requests:
        memory: 20Gi
      limits:
        memory: 20Gi

Attempt to create:

kubectl apply -f test-pod.yaml

Expected error:

Error from server (Forbidden): error when creating "test-pod.yaml": pods "test-memory-exceed" is forbidden: exceeded quota: team-c-quota, requested: limits.memory=20Gi, used: limits.memory=0, limited: limits.memory=16Gi

This proves the quota is working.

Test network policy enforcement

Deploy two pods in different namespaces and check connectivity.

In team-c, run a simple web server pod:

kubectl run web --image=nginx -n team-c
kubectl expose pod web --port=80 -n team-c

From team-b, try to reach it:

kubectl run test-pod --image=busybox -n team-b --restart=Never -- sleep 3600
kubectl exec -n team-b test-pod -- wget -qO- --timeout=5 http://web.team-c.svc.cluster.local

Expected result: the command times out or fails because the default-deny policy in team-c blocks ingress from team-b.

Diagnostic commands for common issues

When something fails, gather information systematically:

1. Check pod status:

kubectl get pods -n team-c

If a pod is in Pending, it might be due to resource quota exhaustion or node pressure.

1. Describe the pod:

kubectl describe pod <pod-name> -n team-c

Look at the Events section for messages like FailedScheduling or Insufficient memory.

1. Check pod logs:

kubectl logs <pod-name> -n team-c --previous

This shows logs from a previous crashed container, useful for diagnosing crash loops.

1. Check deployment rollout status:

kubectl rollout status deployment/<deployment-name> -n team-c

If the rollout is stuck, inspect replicas and events.

1. View resource quota usage:

kubectl describe resourcequota team-c-quota -n team-c

This shows current usage versus limits, helping you see if a tenant is at capacity.

Failure Modes and Recovery

Even with careful planning, things go wrong. Here are common failure modes in Kubernetes multi-tenancy and how to recover.

Quota exhaustion

Symptom: New pods in a tenant namespace stay in Pending state with events like:

FailedScheduling: 0/3 nodes are available: 3 Insufficient cpu.

Or you get an error when creating a pod:

Error from server (Forbidden): pods "my-pod" is forbidden: exceeded quota: team-c-quota

Cause: The tenant has reached its CPU or memory limits.

Recovery:

• Identify which resource is exhausted: kubectl describe resourcequota team-c-quota -n team-c.
• Check actual usage: kubectl top pods -n team-c (if metrics server installed).
• Decide whether to increase the quota or scale down workloads.
• To temporarily increase quota, edit the ResourceQuota:

kubectl edit resourcequota team-c-quota -n team-c

Increase requests.cpu from 4 to 6 and save. But this should be a deliberate action, not a default.

Cross-tenant network access

Symptom: A pod in team-b can reach services in team-a even though you intended isolation.

Cause: Missing or permissive NetworkPolicy, or a policy that allows all ingress.

Recovery:

• List NetworkPolicies in both namespaces:

kubectl get networkpolicy -n team-a
kubectl get networkpolicy -n team-b

• If none exist, apply default-deny policies.
• If policies exist, inspect them:

kubectl describe networkpolicy <policy-name> -n team-a

• Adjust the policy to deny cross-namespace traffic while allowing necessary communication (e.g., from ingress controller).

Overly permissive RBAC

Symptom: A user or service account can list secrets or delete pods in a namespace they should not have access to.

Cause: RoleBinding assigned a ClusterRole like admin or cluster-admin to a broad group.

Recovery:

• Review role bindings:

kubectl get rolebindings -n team-a
kubectl get clusterrolebindings | grep team-a

• Find the subject that should not have access and remove the binding:

kubectl delete rolebinding <binding-name> -n team-a

• Replace with a more restrictive Role or RoleBinding.

Namespace deletion stuck

Symptom: kubectl delete namespace team-c hangs and the namespace remains in Terminating state.

Cause: Finalizers or resources in the namespace that cannot be deleted.

Recovery:

• Check namespace status:

kubectl get namespace team-c -o yaml

• Look for finalizers field. If there is a custom finalizer, you may need to remove it after ensuring cleanup tasks are done.
• List remaining resources:

kubectl get all -n team-c

• Force deletion by patching the namespace to remove finalizers (use with extreme caution):

kubectl patch namespace team-c -p '{"spec":{"finalizers":[]}}' --type=merge

This should only be done when you are sure no external resources depend on it.

Operations Checklist

Use this checklist before, during, and after making multi-tenancy changes. Assign an accountable owner to each item and review the checklist at least monthly, or after any incident.

Pre-change checklist

• [ ] Confirm correct cluster context: kubectl config current-context. Owner: Priya Shah, Engineering Lead. Review: monthly.
• [ ] Backup current RBAC, quotas, and network policies: kubectl get rolebindings,resourcequota,networkpolicy -n <namespace> -o yaml > backup.yaml. Owner: DevOps on-call. Review: weekly.
• [ ] Identify blast radius: which other namespaces or services could be affected? Owner: DevOps on-call. Review: per change.
• [ ] Prepare rollback plan: save original manifests in git. Owner: DevOps on-call. Review: per change.
• [ ] Get approval from tenant owner if change affects their namespace. Owner: Tenant lead (e.g., Team A Lead). Review: per change.

During change

• [ ] Apply to staging namespace first: kubectl apply -f manifest.yaml -n team-c-staging. Owner: DevOps on-call. Review: immediate.
• [ ] Verify expected output: run verification commands listed above. Owner: DevOps on-call. Review: immediate.
• [ ] Monitor cluster-wide metrics for unexpected changes: kubectl top pods --all-namespaces. Owner: SRE on-call. Review: continuous.

Post-change

• [ ] Confirm resource quotas are enforced: try to create a pod that exceeds quota and expect failure. Owner: DevOps on-call. Review: after each change.
• [ ] Test RBAC permissions: use kubectl auth can-i as a test service account. Owner: Security engineer (e.g., Marcus Lee). Review: quarterly.
• [ ] Test network isolation: attempt cross-namespace connection from a test pod. Owner: Security engineer. Review: quarterly.
• [ ] Document any deviations from expected behavior. Owner: DevOps on-call. Review: after each change.
• [ ] Update runbooks with new recovery steps. Owner: Tech writer (e.g., Alex Kim). Review: monthly.

Common Pitfalls and How to Avoid Them

Avoid these frequent mistakes in Kubernetes multi-tenancy.

Pitfall 1: Using a single namespace for all tenants

Why it happens: Easier to manage at first, but leads to resource contention and security issues.

How to avoid: Create a namespace per tenant from day one. Enforce with an admission controller or policy engine.

Recovery: If already in a single namespace, plan a migration. Create new namespaces, set quotas and RBAC, then move workloads gradually.

Pitfall 2: Ignoring resource requests and limits

Why it happens: Teams omit them to simplify YAML, but this allows a noisy neighbor to consume all resources.

How to avoid: Require resource requests and limits via admission policies (e.g., LimitRange). Example LimitRange:

apiVersion: v1
kind: LimitRange
metadata:
  name: default-limits
  namespace: team-c
spec:
  limits:
  - default:
      cpu: 500m
      memory: 512Mi
    defaultRequest:
      cpu: 250m
      memory: 256Mi
    type: Container

Apply and verify: kubectl get limitrange -n team-c.

Recovery: Add LimitRange to existing namespaces and update deployments to include resources.

Pitfall 3: Overly broad RBAC roles

Why it happens: Reusing cluster-admin for convenience.

How to avoid: Follow least privilege. Use Role for namespace-scoped permissions; never bind cluster-admin to a tenant's group.

Recovery: Audit with kubectl get clusterrolebindings -o yaml and remove risky bindings.

Pitfall 4: No network policies

Why it happens: Default Kubernetes allows all pod-to-pod traffic, and many assume isolation is automatic.

How to avoid: Define default-deny policies for each tenant namespace and explicitly allow required traffic.

Recovery: Apply NetworkPolicies and test connectivity as described earlier.

Pitfall 5: Not testing in staging

Why it happens: Pressure to ship quickly leads to direct production changes.

How to avoid: Make staging a requirement in your change process and use CI/CD to enforce it.

Recovery: If a change breaks production, roll back to the previous manifest from version control: kubectl apply -f previous-manifest.yaml after reverting in git.

Conclusion

Kubernetes multi-tenancy commands are only useful when they are version-scoped, observable, and reversible. Copying commands without understanding prerequisites and expected output is not a reliable operational practice.

Start with one low-risk verification: choose a tenant namespace, record its current state, run the documented checks, compare the result with the expected signal, and review dependencies like Namespace, Resource Quota, and RBAC.

A reliable workflow makes failure visible, protects sensitive values, limits changes to the intended resource, and defines recovery verification before an incident forces the decision. By applying the commands and checklists in this article, you can maintain a stable and secure multi-tenant cluster.