Kubernetes Authentication: Essential Commands for Secure Cluster Access

Introduction

Kubernetes authentication is the first gate every request must pass through before it reaches the API server. Whether you are a developer debugging a failing deployment, a platform engineer onboarding a new team, or an SRE investigating an unusual permission error, knowing the exact commands to inspect and verify authentication is essential for safe, efficient cluster operations.

This article provides a practical command reference for Kubernetes authentication. It covers core concepts, common workflows, and diagnostic techniques, with concrete examples and expected outputs. We focus on commands that are safe to run in a production cluster, emphasize least privilege, and include recovery steps for common failure modes.

The goal is operational clarity: understand who is authenticated, how they are authenticated, and whether their credentials are working as intended. You will learn to inspect certificates, tokens, service accounts, and validate Role-Based Access Control (RBAC) configurations with a few well-chosen commands.

Version and Environment Inventory

Before running authentication commands, know your cluster version and components. Authentication behavior can vary between Kubernetes versions, especially with features like TokenRequest API (stable since 1.20) and bound service account tokens (default since 1.21). Use the following read-only commands to gather information without changing state.

kubectl version --short

Example output:

Client Version: v1.27.3
Kustomize Version: v5.0.1
Server Version: v1.27.3

This confirms the client and server are compatible. If the server version is significantly older or newer than expected, some authentication features may not be available.

Check the API server's authentication configuration by viewing its pod specification (if you have access to the control plane):

kubectl -n kube-system get pod kube-apiserver-<master-node> -o yaml | grep -A5 'command:'

Look for flags like --authentication-token-webhook-config-file, --oidc-issuer-url, or --client-ca-file. These indicate which authentication methods are enabled.

Practical environment checks

• Verify you can reach the API server and that your current credentials work:

  kubectl auth whoami

Output example:

  ATTRIBUTE   VALUE
  Username    kubernetes-admin
  UID         abc123... (on some clusters)
  Groups      [system:masters system:authenticated]

If you get Error from server (Forbidden), your user lacks the selfsubjectaccessreviews permission, but basic authentication still works.

• List nodes to confirm cluster health:

  kubectl get nodes

A NotReady node may indicate certificate or kubelet authentication issues.

• Check the default namespace's service accounts:

  kubectl get serviceaccounts

Each namespace has a default service account. Problems with this account often cause authentication failures for workloads.

Keep all commands read-only until you have established a baseline. Use kubectl with --dry-run=client for any change to understand what resources would be modified.

Understanding Kubernetes Authentication

Kubernetes does not manage user accounts natively. Instead, it relies on external identity providers (X.509 certificates, static tokens, OpenID Connect, webhooks) for human users, and service accounts for processes running inside the cluster. All authenticated requests are then authorized separately, usually via RBAC.

Core commands for authentication inspection

1. Inspect your current credentials

Your kubeconfig file stores cluster, user, and context information. Use kubectl config view to see it:

kubectl config view --minify

This shows the current context, cluster, and user details. Example:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: REDACTED
    server: https://192.168.1.100:6443
  name: my-cluster
contexts:
- context:
    cluster: my-cluster
    user: my-user
  name: my-context
current-context: my-context
kind: Config
preferences: {}
users:
- name: my-user
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED

To see the decoded certificate details (expiry, subject, issuer):

kubectl config view --raw -o jsonpath='{.users[?(@.name=="my-user")].user.client-certificate-data}' | base64 -d | openssl x509 -noout -subject -issuer -dates

Output may look like:

subject=O=system:masters, CN=kubernetes-admin
issuer=CN=kubernetes
notBefore=Jan 1 00:00:00 2023 GMT
notAfter=Dec 31 23:59:59 2024 GMT

Check the expiration date. An expired certificate causes authentication failures.

2. Check service account tokens

Each service account is associated with one or more secrets containing tokens. List them:

kubectl get secrets

Look for secrets of type kubernetes.io/service-account-token. For a specific service account default:

kubectl get secret $(kubectl get serviceaccount default -o jsonpath='{.secrets[0].name}') -o yaml

The token is base64-encoded. Decode it to inspect the JWT:

kubectl get secret <secret-name> -o jsonpath='{.data.token}' | base64 -d

This will show the JWT. You can decode the payload to see claims:

# Copy the token and paste it into jwt.io, or use a command-line tool

In Kubernetes 1.24+, legacy service account token secrets are no longer automatically created. Use TokenRequest API to generate a short-lived token:

kubectl create token default

This produces a token valid for 1 hour by default. Example output:

eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJhdWQiOlsiaHR0cHM6Ly9rdWJlcm5ldGVzLmRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwiXSwiZXhwIjoxNjg4NzY1NDMyLCJpYXQiOjE2ODg3NjE4MzIsImlzcyI6Imh0dHBzOi8va3ViZXJuZXRlcy5kZWZhdWx0LnN2Yy5jbHVzdGVyLmxvY2FsIiwia3ViZXJuZXRlcy5pbyI6eyJuYW1lc3BhY2UiOiJkZWZhdWx0Iiwic2VydmljZWFjY291bnQiOiJ7XCJuYW1lXCI6XCJkZWZhdWx0XCIsXCJ1aWQiXCI6XCI5ZDc5YzA1MS05YjQxLTQ0MjEtOGVjMC0wM2ExMjI2OTk2YmZcIn0ifSwibmJmIjoxNjg4NzYxODMyLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGVmYXVsdDpkZWZhdWx0In0.eyJhbGciOiJSUzI1NiJ9... (truncated)

3. Verify RBAC permissions

Use kubectl auth can-i to check if a specific action is allowed for the current user or a service account.

Check if you can create pods in the current namespace:

kubectl auth can-i create pods

Output: yes or no.

For a service account my-sa in namespace my-ns, check if it can list deployments:

kubectl auth can-i list deployments --as=system:serviceaccount:my-ns:my-sa -n my-ns

To see who can perform an action across the cluster, use kubectl auth reconcile but that is for applying RBAC changes. For troubleshooting, inspect roles and bindings.

Common Authentication Workflows with Examples

Adding a new user with client certificates

1. Generate a private key and certificate signing request (CSR):

   openssl genrsa -out user.key 2048
   openssl req -new -key user.key -out user.csr -subj "/CN=alice/O=developers"

The CN is the username, and O becomes the group.

1. Create a Kubernetes CertificateSigningRequest resource:

   cat <<EOF | kubectl apply -f -
   apiVersion: certificates.k8s.io/v1
   kind: CertificateSigningRequest
   metadata:
     name: alice-csr
   spec:
     request: $(cat user.csr | base64 | tr -d '\n')
     signerName: kubernetes.io/kube-apiserver-client
     usages:
     - client auth
   EOF

1. Approve the CSR:

   kubectl certificate approve alice-csr

1. Retrieve the signed certificate:

   kubectl get csr alice-csr -o jsonpath='{.status.certificate}' | base64 -d > user.crt

1. Add the user to kubeconfig:

   kubectl config set-credentials alice --client-certificate=user.crt --client-key=user.key
   kubectl config set-context alice-context --cluster=my-cluster --user=alice
   kubectl config use-context alice-context

1. Test access:

   kubectl get pods

If no RBAC permissions are assigned, output will be Error from server (Forbidden): pods is forbidden: User "alice" cannot list resource "pods" in API group "" in the namespace "default".

1. Grant permissions using a Role and RoleBinding:

   kubectl create role pod-reader --verb=get,list,watch --resource=pods
   kubectl create rolebinding alice-pod-reader --role=pod-reader --user=alice

Now kubectl get pods should succeed.

Creating a service account for an application

1. Create a dedicated service account:

   kubectl create serviceaccount my-app-sa

1. Assign it to a deployment:

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: my-app
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: my-app
     template:
       metadata:
         labels:
           app: my-app
       spec:
         serviceAccountName: my-app-sa
         containers:
         - name: app
           image: nginx

1. Verify the pod is using the service account:

   kubectl get pod -l app=my-app -o jsonpath='{.items[0].spec.serviceAccountName}'

Output: my-app-sa.

1. Grant the service account access to needed resources. For example, allow it to read configmaps in its namespace:

   kubectl create role configmap-reader --verb=get,list --resource=configmaps
   kubectl create rolebinding my-app-sa-configmap-reader --role=configmap-reader --serviceaccount=default:my-app-sa

Using OpenID Connect (OIDC) authentication

If your cluster is configured with OIDC, users obtain an ID token from the identity provider and use it as a bearer token. The API server validates the token against the provider's configuration.

To use an OIDC token, provide it directly in the kubeconfig:

kubectl config set-credentials oidc-user --auth-provider=oidc --auth-provider-arg=idp-issuer-url=https://issuer.example.com --auth-provider-arg=client-id=kubernetes --auth-provider-arg=id-token=<token>

Or you can pass the token with each request:

kubectl --token=<token> get pods

To inspect the token claims, decode the JWT payload. The sub claim becomes the username, and groups claim becomes group memberships.

Verification and Diagnostics

Diagnosing authentication failures

Authentication failures typically result in HTTP 401 errors from the API server. Common messages:

• Unauthorized
• error: You must be logged in to the server (Unauthorized)
• x509: certificate signed by unknown authority

Use kubectl with increased verbosity to see detailed request/response:

kubectl get pods -v=8

Look for lines containing 401 or Authorization.

If using a token, check its expiration and signature:

kubectl create token default --duration=0s  # In older versions, may fail; use manual decode.

Decode the token with a tool like jq and base64:

# Extract payload (second segment)
TOKEN=$(kubectl create token default)
echo $TOKEN | cut -d'.' -f2 | base64 -d 2>/dev/null | jq .

Example output:

{
  "aud": ["https://kubernetes.default.svc.cluster.local"],
  "exp": 1688765432,
  "iat": 1688761832,
  "iss": "https://kubernetes.default.svc.cluster.local",
  "kubernetes.io": {
    "namespace": "default",
    "serviceaccount": {"name":"default","uid":"9d79c051-..."}
  },
  "nbf": 1688761832,
  "sub": "system:serviceaccount:default:default"
}

Check that exp is in the future and aud includes the API server audience.

Verifying RBAC policies

To see which roles are bound to a user or service account:

kubectl get rolebindings,clusterrolebindings --all-namespaces -o custom-columns='KIND:.kind,NAMESPACE:.metadata.namespace,NAME:.metadata.name,ROLE:.roleRef.name,SUBJECTS:.subjects[*].name' | grep alice

For a deeper analysis, use kubectl auth reconcile --dry-run=client -f rbac.yaml to see differences without applying.

Checking service account token mounts

Inside a pod, the service account token is mounted at /var/run/secrets/kubernetes.io/serviceaccount/token. To verify the token is the expected one:

kubectl exec <pod-name> -- cat /var/run/secrets/kubernetes.io/serviceaccount/token

Compare with the token from the secret or TokenRequest.

Failure Modes and Recovery

Understanding common authentication failures helps you respond quickly.

Expired client certificate

Symptom: When running kubectl get pods, you see:

Unable to connect to the server: x509: certificate has expired or is not yet valid

Why it happens: The client certificate in kubeconfig has passed its notAfter date. Recovery:

1. Check expiry with openssl x509 -noout -dates -in <cert-file>.
2. Obtain a new certificate from your CA or regenerate via CSR as described earlier.
3. Update kubeconfig credentials.

Invalid or expired token

Symptom: API requests return 401 Unauthorized: invalid token. Why it happens: The token may be expired, malformed, or the signing key has rotated. Recovery:

• For service account tokens: generate a new token with kubectl create token <sa-name> and update your application.
• For static tokens: check the API server's --token-auth-file and ensure the token matches.
• For OIDC: refresh the ID token with the provider.

Certificate signed by unknown authority

Symptom: x509: certificate signed by unknown authority when connecting. Why it happens: The cluster's CA certificate in kubeconfig does not match the server's certificate, or the CA is not trusted. Recovery:

• Ensure the certificate-authority-data or certificate-authority in kubeconfig points to the correct CA.
• If using a custom CA, add it to your system trust store or specify insecure-skip-tls-verify: true (not recommended for production).

User cannot perform action despite role binding

Symptom: kubectl auth can-i create pods returns no but you believe the user should have permission. Why it happens: RBAC is misconfigured: role binding in wrong namespace, wrong subject (user vs service account), or missing verbs/resources. Recovery:

• List bindings with kubectl get rolebindings -n <namespace> -o yaml and verify subjects.
• Check the role's rules: kubectl get role <role> -o yaml.
• Use kubectl auth can-i --list --as=<user> to see all permissions for that user.

Service account not mounting token

Symptom: Application inside pod fails to authenticate to API server, and the token file is missing or empty. Why it happens: The pod's service account name may be misspelled, or the service account does not exist. Recovery:

• Verify service account: kubectl get serviceaccounts -n <namespace>.
• Check pod spec: kubectl get pod <pod> -o jsonpath='{.spec.serviceAccountName}'.
• If missing, recreate pod with correct serviceAccountName.

Operations Checklist

Use this checklist to ensure authentication is properly managed in your cluster. Assign an owner and frequency for review.

Key operational practices

1. Use RBAC with least privilege: Grant only necessary permissions. Avoid using cluster-admin for daily work.
2. Prefer short-lived tokens: For service accounts, use TokenRequest API or projected tokens with expiration. For users, use OIDC with refresh tokens.
3. Monitor authentication metrics: Enable API server audit logs and alert on 401 surges.
4. Automate certificate renewal: Use cert-manager or a similar controller for client certificates.
5. Document recovery steps: Maintain runbooks for expired certificates, token issues, and RBAC misconfigurations.

Common Pitfalls and How to Avoid Them

Pitfall 1: Using the default service account for all workloads

Why it happens: It is convenient because every namespace has one, and developers may not think about identity isolation. How to avoid: Create a dedicated service account for each application and explicitly set serviceAccountName in pod specs. This limits blast radius if a token is compromised. Recovery: If already using default, create new service accounts, update deployments, and delete default token mounts with automountServiceAccountToken: false where not needed.

Pitfall 2: Overly broad RBAC permissions with wildcards

Why it happens: Developers often request * on resources to get things working quickly, or admins copy cluster-admin role for simplicity. How to avoid: Start with minimal permissions, test with kubectl auth can-i --list, and expand only when necessary. Use tools like kubectl-who-can or audit2rbac to generate least-privilege roles. Recovery: Review all role bindings for wildcard usage with kubectl get clusterrolebindings -o json | jq '.items[] | select(.roleRef.name=="cluster-admin")' and tighten.

Pitfall 3: Hardcoding service account tokens in application code or config files

Why it happens: It seems easier than mounting from secrets, especially when running outside Kubernetes. How to avoid: Use the mounted token file or TokenRequest API within the cluster. Outside, use workload identity or OIDC. Recovery: Remove hardcoded tokens, rotate them immediately, and adopt secure token delivery.

Pitfall 4: Ignoring certificate expiry

Why it happens: Client certificates have long lifetimes, and no alerting is set up. How to avoid: Monitor certificate expiry with Prometheus alerts or cert-manager notifications. Recovery: Implement automated renewal and have a manual process as backup.

Pitfall 5: Not differentiating between authentication and authorization

Why it happens: Users often confuse the two: they see Forbidden (403) and think authentication is broken. How to avoid: Diagnose with kubectl auth can-i. If whoami succeeds but can-i returns no, it is an authorization (RBAC) issue, not authentication. Recovery: Fix the role binding rather than trying to change user credentials.

Pitfall 6: Running with cluster-admin for CI/CD pipelines

Why it happens: CI/CD needs broad access to deploy across namespaces, and granting cluster-admin is easy. How to avoid: Create dedicated service accounts with narrowly scoped roles for each pipeline. Use Kubernetes namespace isolation. Recovery: Audit pipeline credentials, create new roles, and rotate tokens.

Conclusion

Authentication is the foundation of Kubernetes security. Mastering the commands to inspect and manage authentication ensures that you can quickly resolve access issues, maintain least privilege, and keep your cluster secure. This article provided a comprehensive reference: from checking your environment and credentials, to creating users and service accounts, to diagnosing failures and avoiding common pitfalls.

Start by applying one low-risk verification from this guide: run kubectl auth whoami to see who you are authenticated as, then check your permissions with kubectl auth can-i --list. Document the results and compare them with expected access. Then review your service accounts and token practices. Over time, expand to automated checks and audits.

A robust authentication operation system makes failures visible, protects sensitive credentials, limits changes to intended resources, and ensures recovery plans are tested before an incident occurs.