Intro
Kubernetes RBAC (Role-Based Access Control) is how you grant or restrict permissions within a cluster. A Role defines a set of permissions within a namespace, such as "read pods" or "create deployments." A RoleBinding then connects that Role to a user, group, or service account. While this concept is simple, operating these resources safely requires a structured approach: observe current state, make minimal changes, and verify the outcome with expected output and clear failure signals.
This guide focuses on Kubernetes Role commands for developers, DevOps consultants, and technical startup teams. It walks through observing roles and bindings, creating and modifying them, diagnosing access issues, and recovering from mistakes—all with concrete kubectl commands and example outputs. The goal is operational safety: understand what exists before changing it, limit the blast radius, avoid hard‑coding secrets, and always have a recovery path.
Version and Environment Inventory
Before changing any RBAC configuration, establish a baseline of your cluster, the relevant API versions, and the current roles and bindings. This prevents wrong assumptions and gives you a rollback point.
Check cluster and kubectl version
Commands:
kubectl version --short
Example output:
Client Version: v1.25.3
Kustomize Version: v4.5.7
Server Version: v1.25.3
This tells you the API version your client and server speak. RBAC resources (Role, RoleBinding, ClusterRole, ClusterRoleBinding) are in the rbac.authorization.k8s.io/v1 API group. Older clusters may use v1beta1, so confirm with:
kubectl api-versions | grep rbac
Expected output includes:
rbac.authorization.k8s.io/v1
If only v1beta1 appears, your cluster is older and some v1 features may not be supported.
List current roles and bindings
See all Roles in the current namespace:
kubectl get roles
Example:
NAME CREATED AT
pod-reader 2023-03-15T10:30:00Z
deployer 2023-03-16T09:00:00Z
To list Roles in all namespaces, add -A:
kubectl get roles -A
Similarly, list RoleBindings:
kubectl get rolebindings
For a detailed view of a specific Role, use describe (read-only, safe):
kubectl describe role pod-reader
Example output snippet:
Name: pod-reader
Namespace: default
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
pods [] [] [get, watch, list]
This tells you exactly what permissions the Role grants. In this case, it allows reading pods in the default namespace.
Identify service accounts and subjects
RBAC subjects are often service accounts. List service accounts in the namespace:
kubectl get serviceaccounts
Example:
NAME SECRETS AGE
default 0 30d
my-app 0 10d
If a RoleBinding references a service account, you can see that in its describe output:
kubectl describe rolebinding read-pods
Relevant lines:
Role:
Kind: Role
Name: pod-reader
Subjects:
Kind Name Namespace
---- ---- ---------
ServiceAccount my-app default
This association is critical for troubleshooting access issues.
Practical checklist for environment inventory
- [x] Cluster and kubectl versions are compatible and support
rbac.authorization.k8s.io/v1. - [x] You have listed existing Roles and RoleBindings in the target namespace.
- [x] You have described the specific Role involved and noted its rules.
- [x] You have identified the subject (user, group, or service account) that needs access.
- [x] You have backed up existing resource manifests using
kubectl get role <name> -o yaml > role-backup.yamlandkubectl get rolebinding <name> -o yaml > rb-backup.yaml.
Always create backups before modifying RBAC, because a bad change can lock out users or break applications.
Safe Configuration Path
When changing Role definitions or bindings, follow a minimal-change approach: prefer updating an existing resource over replacing it, and test in a non-production namespace first.
Creating a new Role
Define the Role in a YAML file. Example pod-reader-role.yaml:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: default
name: pod-reader
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["pods"]
verbs: ["get", "watch", "list"]
Apply it:
kubectl apply -f pod-reader-role.yaml
Expected output:
role.rbac.authorization.k8s.io/pod-reader created
Verify:
kubectl get role pod-reader
Output:
NAME CREATED AT
pod-reader 2023-03-16T15:00:00Z
Creating a RoleBinding
Create read-pods-binding.yaml:
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: read-pods
namespace: default
subjects:
- kind: ServiceAccount
name: my-app
namespace: default
roleRef:
kind: Role
name: pod-reader
apiGroup: rbac.authorization.k8s.io
Apply:
kubectl apply -f read-pods-binding.yaml
Output:
rolebinding.rbac.authorization.k8s.io/read-pods created
Verify:
kubectl get rolebinding read-pods
Output:
NAME ROLE AGE
read-pods Role/pod-reader 10s
Editing an existing Role
Use kubectl edit only when you fully understand the impact. For small changes, patch is safer because it is more explicit. Example: add "create" verb to the pod-reader Role.
kubectl patch role pod-reader --type='json' -p='[{"op": "add", "path": "/rules/0/verbs", "value": ["get", "watch", "list", "create"]}]'
Verify:
kubectl describe role pod-reader
Now the verbs include create.
Alternatively, if you prefer to edit the file and reapply:
kubectl apply -f pod-reader-role.yaml
apply will merge changes (if you use the right fields) and show role.rbac.authorization.k8s.io/pod-reader configured.
Simulating permissions with auth can-i
Before deploying a change, test what a subject can do using kubectl auth can-i. This is an excellent read-only way to validate RBAC.
Check if the my-app service account can list pods in default namespace:
kubectl auth can-i list pods --as=system:serviceaccount:default:my-app -n default
Expected output: yes or no.
This command impersonates the service account and runs an authorization check without actually performing the action. It is invaluable for troubleshooting.
Example: after adding the create verb, verify:
kubectl auth can-i create pods --as=system:serviceaccount:default:my-app -n default
If it returns yes, the change is effective.
Safe rollout practices
- Use a dedicated test namespace (e.g.,
rbac-test) to try new Roles and Bindings before applying to production. - Prefer
kubectl createorkubectl applywith explicit files; avoidkubectl editon live systems without review. - Always set
--dry-run=client -o yamlto see the resulting object without persisting it:
kubectl create role test-role --verb=get,list --resource=pods --dry-run=client -o yaml > test-role.yaml
Then inspect the file, and if fine, apply it.
- Use namespaced Roles (not ClusterRoles) unless you truly need cluster-wide access. This limits the blast radius.
Verification and Diagnostics
After making changes, verify that the configuration is correct and that the intended subjects actually have the permissions you expect. Use both resource inspection and actual impersonated requests.
Inspect RBAC resources
Get roles and bindings with output formats for easier reading:
kubectl get roles -o wide
Example:
NAME CREATED AT
pod-reader 2023-03-16T15:00:00Z
deployer 2023-03-17T09:00:00Z
wide may show extra columns depending on the version.
To see all rules across all roles in a namespace, use:
kubectl get roles -o yaml
And for rolebindings, see the subjects and roleRef:
kubectl get rolebindings -o yaml
Test actual access
Use kubectl auth can-i with different subjects and verbs. For a user group, you can use --as-group. Example: check if a user in the dev group can delete pods:
kubectl auth can-i delete pods --as=someuser --as-group=dev -n default
If it returns no, you know the role lacks that permission (or the binding is wrong).
Test with a service account as shown earlier. Also test denied actions to confirm restrictions are in place:
kubectl auth can-i update deployments --as=system:serviceaccount:default:my-app -n default
Expected: no (if not granted).
Check audit events
For deeper diagnostics, use kubectl get events --all-namespaces to see recent events, but RBAC denials may not show there. If your cluster has audit logging enabled, you can inspect apiserver audit logs for lines mentioning system:serviceaccount:default:my-app and forbidden.
Example command (on control-plane node or via logging stack):
grep "forbidden" /var/log/kubernetes/audit.log | grep "my-app" | tail -20
This often reveals the exact resource and verb denied.
Practical diagnostic workflow
- Identify the subject (user, group, SA) that is blocked.
- Run
kubectl auth can-i <verb> <resource> --as=<subject> -n <namespace>. - If
no, check the RoleBinding to see if the subject is included.
kubectl describe rolebinding <binding-name>
- If binding looks correct, describe the Role to see if the verb/resource is granted.
kubectl describe role <role-name>
- If the rule is missing, add it and re-test with
auth can-i. - If still denied, check for any cluster-level DNS or network policies, or if the subject has overridden permissions via another binding.
Failure Modes and Recovery
RBAC mistakes can lock out users, break applications, or open security holes. Recognize common failure modes and know how to recover quickly.
Common failure scenarios
1. Missing permission for an application Symptom: Application logs show Error: pods is forbidden: User "system:serviceaccount:default:my-app" cannot list resource "pods" in API group "" in the namespace "default". Diagnosis: The Role or RoleBinding does not grant the needed permission. Recovery: Add the missing rule to the Role, or create a new Role and bind it. Verify with auth can-i.
2. Over-permissive Role Symptom: Security audit flags that a Role grants * on secrets. Diagnosis: The Role's rules are too broad. Recovery: Edit the Role to restrict verbs and resources. Example:
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
resourceNames: ["specific-secret"] # only that secret
Apply and verify.
3. Accidental deletion of RoleBinding Symptom: Users suddenly lose access. Diagnosis: kubectl get rolebinding <name> returns NotFound. Recovery: If you have a backup YAML (as recommended), reapply:
kubectl apply -f rb-backup.yaml
If no backup, recreate the binding from documentation or from another namespace's similar binding.
4. Subject not matching due to namespace Symptom: A service account in staging cannot access resources in production, even though a RoleBinding exists. Diagnosis: RoleBinding is namespaced; a RoleBinding in production cannot grant access to a service account in staging unless the binding explicitly references it (subject namespace can differ, but the binding itself is in the resource's namespace). Recovery: Ensure the RoleBinding is created in the namespace of the resource, and the subject namespace is correctly specified.
5. API version mismatch Symptom: error: unable to recognize "role.yaml": no matches for kind "Role" in version "rbac.authorization.k8s.io/v1beta1". Diagnosis: Cluster is newer and no longer serves v1beta1. Recovery: Change apiVersion to rbac.authorization.k8s.io/v1 and reapply.
Recovery best practices
- Always keep YAML backups before changes:
kubectl get role <name> -o yaml > role-backup-$(date +%F).yaml. - Use
kubectl apply --record(deprecated in some versions) or version control for manifests. - Test changes in a non-production namespace first.
- Have an emergency break-glass account with cluster-admin privileges (or at least permissions to edit RBAC) for recovery.
- Document the expected behavior and a rollback plan for every RBAC change.
Simulating failure to build confidence
You can intentionally create a misconfigured binding to see how errors appear and practice recovery. Example: Create a Role with no verbs (invalid):
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: empty-role
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: []
Applying this will result in an error: spec.rules[0].verbs: Required value: at least one verb must be specified. This teaches you about required fields.
Operations Checklist
Use this checklist for day-to-day RBAC operations. Each item includes a concrete command or check with expected result.
Before any change
- [ ] Check cluster version:
kubectl version --short- ensure RBAC v1 is supported. - [ ] List existing roles and bindings in target namespace:
kubectl get roles,rolebindings -n <namespace>. - [ ] Describe the specific Role involved:
kubectl describe role <role-name> -n <namespace>. - [ ] Identify the subject requiring access: user, group, or service account.
- [ ] Backup current RBAC resources:
kubectl get role <name> -o yaml > backup/role-<name>-$(date +%F).yamland same for rolebinding.
Creating or modifying roles
- [ ] Write manifest with least privilege: only necessary API groups, resources, and verbs.
- [ ] Dry-run the creation:
kubectl create role test --verb=get,list --resource=pods --dry-run=client -o yaml > test.yamland inspect. - [ ] Apply manifest:
kubectl apply -f role.yaml- expectedrole.rbac.authorization.k8s.io/<name> createdorconfigured. - [ ] Verify new role:
kubectl get role <name> -o yaml.
Creating or modifying bindings
- [ ] Ensure subject namespace is correct (if ServiceAccount).
- [ ] Apply binding:
kubectl apply -f binding.yaml. - [ ] Verify binding:
kubectl get rolebinding <name> -o yaml.
Verification
- [ ] Test access with impersonation:
kubectl auth can-i <verb> <resource> --as=system:serviceaccount:<namespace>:<sa-name> -n <namespace>- expectyesorno. - [ ] Test a denied action to confirm least privilege:
kubectl auth can-i delete secrets --as=...- expectno. - [ ] Check application logs or run a quick pod with the service account to test actual behavior (if needed).
Post-change
- [ ] Update documentation (runbooks, comments in manifests).
- [ ] Store final manifests in version control.
- [ ] Notify relevant team members about the access change.
Emergency recovery
- [ ] If lockout occurs, use break-glass admin account or alternate path (e.g., direct API access with client certificate) to restore RBAC.
- [ ] Reapply last known good manifests.
- [ ] Verify restored access with
auth can-iand actual user test. - [ ] Conduct post-incident review: why did it happen? How to prevent?
Conclusion
Kubernetes Role commands are essential for managing access within namespaces, but they demand careful operational discipline. By following the structured approach outlined here—inventory, safe configuration, verification, failure recovery, and checklists—you can avoid common pitfalls like over-privileged roles, broken bindings, and lockouts.
As a next step, pick one low-risk scenario: create a read-only Role for a service account in a test namespace, verify with kubectl auth can-i, and then intentionally deny an action to see the failure mode. Record the commands and results in your team's runbook. This hands-on practice builds confidence and reinforces safe habits.
Reliable RBAC management is about observability, least privilege, and reversibility. With the right commands and a methodical approach, you can keep your cluster secure and your applications running smoothly.