Kubernetes PersistentVolumes, StorageClasses, and PVCs: A Practical Implementation Guide

Intro

Running containers in production is straightforward, but operating stateful workloads reliably requires a solid grasp of Kubernetes storage primitives. This guide walks through PersistentVolumes (PV), PersistentVolumeClaims (PVC), and StorageClasses with real manifests, verification commands, and the most common failure modes you will encounter when moving from a laptop cluster to a production‑like environment.

The target audience includes developers, DevOps consultants, and technical startup teams who need to move from concept to local verification quickly. By the end you will be able to create a PV, request it via a PVC, let a StorageClass provision it dynamically, and confirm that a Pod can read and write data that survives Pod restarts.

Workflow Overview

1. Identify the resources you will create

2. Verify each step with kubectl

# List all PVs and their status
kubectl get pv -o wide

# List PVCs in the current namespace
kubectl get pvc -o wide

# Show StorageClasses available in the cluster
kubectl get storageclass

# Describe a specific PVC to see binding events
kubectl describe pvc my-data-pvc

# Describe the bound PV
kubectl describe pv <pv-name>

# Check Pod events and mount status
kubectl describe pod my-app-xyz

3. Common failure scenarios and how to spot them

Local Pilot Plan

1. Set up a minimal cluster (kind / minikube / k3d)

# Example with kind
kind create cluster --name storage-demo
kubectl cluster-info --context kind-storage-demo

2. Install a local CSI driver for dynamic provisioning (optional but realistic)

# Hostpath CSI driver for kind
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-host-path/master/deploy/kubernetes-1.24/deploy.yaml

3. Create a StorageClass that uses the hostpath driver

# storageclass-hostpath.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: local-hostpath
provisioner: hostpath.csi.k8s.io
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Delete
parameters:
  type: directory

Apply and verify:

kubectl apply -f storageclass-hostpath.yaml
kubectl get storageclass local-hostpath

4. Request storage with a PVC

# pvc-data.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi
  storageClassName: local-hostpath

kubectl apply -f pvc-data.yaml
kubectl get pvc data-pvc -w   # watch until STATUS becomes Bound

5. Deploy a workload that mounts the PVC

# deployment-app.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: storage-writer
spec:
  replicas: 1
  selector:
    matchLabels:
      app: storage-writer
  template:
    metadata:
      labels:
        app: storage-writer
    spec:
      containers:
        - name: writer
          image: busybox:1.36
          command: [
            "sh",
            "-c",
            "while true; do echo $(date) >> /data/out.txt; sleep 5; done"
          ]
          volumeMounts:
            - name: data-volume
              mountPath: /data
      volumes:
        - name: data-volume
          persistentVolumeClaim:
            claimName: data-pvc

kubectl apply -f deployment-app.yaml
kubectl rollout status deployment/storage-writer

6. Validate write and persistence

# Get the pod name
POD=$(kubectl get pods -l app=storage-writer -o jsonpath='{.items[0].metadata.name}')

# Inspect the file inside the container
kubectl exec $POD -- cat /data/out.txt

# Delete the pod to force recreation
kubectl delete pod $POD

# Wait for new pod and verify data survived
kubectl rollout status deployment/storage-writer
NEW_POD=$(kubectl get pods -l app=storage-writer -o jsonpath='{.items[0].metadata.name}')
kubectl exec $NEW_POD -- cat /data/out.txt

If the timestamps continue without gaps, the PV survived the Pod restart.

7. Simulate a failure: change the PVC to request ReadWriteMany

# pvc-data-rwm.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data-pvc-rwm
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 2Gi
  storageClassName: local-hostpath

kubectl apply -f pvc-data-rwm.yaml
kubectl get pvc data-pvc-rwm

The PVC will stay Pending because the hostpath driver only supports ReadWriteOnce. The describe output shows FailedBinding with a message about incompatible access modes — exactly the scenario you will hit when moving to a network filesystem that does support ReadWriteMany.

Conclusion

Treating storage configuration as testable code — not as a one‑time copy‑paste — is the safest way to avoid surprises in CI/CD pipelines and production clusters. The workflow above lets you:

1. Define a StorageClass that matches your infrastructure (local hostpath, cloud CSI, NFS, etc.).
2. Request storage with a PVC that expresses the exact capacity and access mode you need.
3. Verify binding with kubectl get pvc and kubectl describe pvc.
4. Mount the PVC in a real workload and confirm data survives Pod recreation.
5. Intentionally break the request (wrong access mode, missing StorageClass, exhausted quota) to see the exact error messages you will later troubleshoot under pressure.

Next steps for a production‑grade rollout:

• Replace the hostpath driver with the CSI driver of your cloud provider (EBS, PD, Azure Disk, etc.).
• Set volumeBindingMode: WaitForFirstConsumer for topology‑aware provisioning.
• Use reclaimPolicy: Retain for critical data and implement a backup strategy (Velero, CSI snapshots).
• Add fsGroup and runAsUser in the Pod securityContext to avoid permission issues on shared volumes.

By running the commands locally first, you turn abstract concepts into observable behavior, making the transition to staging and production predictable rather than hopeful.