Kubernetes Certificate Signing Request CI/CD Automation with Practical Examples

Intro

Automating Kubernetes Certificate Signing Request (CSR) approval in CI/CD pipelines is a common need for teams that manage their own TLS certificates or integrate with external PKI. Without a controlled workflow, operators either approve CSRs manually (slow and error-prone) or leave them pending indefinitely, breaking application deployments.

This article provides practical guidance for developers, DevOps consultants, and technical startup teams who want to automate CSR approval safely. It covers the lifecycle of a CSR in Kubernetes, how to configure RBAC for automation, how to build a CI/CD step that approves CSRs, how to verify the issued certificate, and how to handle failures and rollbacks.

Throughout, we emphasize operational safety: observe before changing, limit the blast radius, protect sensitive values, verify results, and document recovery paths.

Version and Environment Inventory

Before automating CSR approval, establish the exact Kubernetes version and environment details. The behavior of CertificateSigningRequest resources changed over releases, especially around signer names and approval permissions.

Key prerequisites:

• A Kubernetes cluster (v1.19 or later recommended; signer-specific approval introduced earlier but matured later)
• kubectl configured with appropriate permissions to inspect CSRs
• If using a custom signer, ensure it is installed and configured (e.g., cert-manager, kube-controller-manager's built-in signer, or an external CA)
• A service account for CI/CD automation with minimal RBAC

Start with read-only observation:

kubectl version --short
kubectl get csr

Expected output includes the cluster version and a list of CSRs with their conditions. For example:

NAME        AGE   SIGNERNAME                     REQUESTOR          CONDITION
my-app-csr  2m    kubernetes.io/kube-apiserver-client   system:serviceaccount:default:my-app   Pending

Note the signerName. Modern clusters use signer names like kubernetes.io/kube-apiserver-client, kubernetes.io/kubelet-serving, or custom ones. Automation must match the signer to the intended use; approving a CSR for the wrong signer can issue a certificate with unintended permissions.

Check component health:

kubectl get pods -n kube-system | grep -E 'kube-controller-manager|kube-apiserver'

If the controller manager that runs the CSR approver is unhealthy, automation may appear to run but not complete.

For local testing, use a single-node cluster (kind, minikube) and apply one CSR manifest at a time. Verify with:

kubectl get csr my-csr -o yaml

Before moving to production, confirm the cluster's certificate API is enabled (it is by default) and that the API server has --cluster-signing-cert-file and --cluster-signing-key-file set if using the built-in signer.

Safe Configuration Path

Automating CSR approval requires careful RBAC configuration. The principle of least privilege is critical: a CI/CD service account should be able to approve only CSRs with specific signer names and, ideally, only for its namespace or a known naming pattern.

Step 1: Create a dedicated ServiceAccount

apiVersion: v1
kind: ServiceAccount
metadata:
  name: csr-approver
  namespace: ci-cd

Apply it:

kubectl apply -f sa.yaml

Step 2: Define a ClusterRole with scoped approval permissions

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: csr-approver
rules:
- apiGroups: ["certificates.k8s.io"]
  resources: ["certificatesigningrequests/approval"]
  verbs: ["update"]
- apiGroups: ["certificates.k8s.io"]
  resources: ["certificatesigningrequests"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["certificates.k8s.io"]
  resources: ["certificatesigningrequests/status"]
  verbs: ["patch", "update"]

This allows viewing CSRs and updating the approval subresource, which is distinct from editing the CSR itself.

Step 3: Bind the ClusterRole to the ServiceAccount with a resourceName restriction if possible

If you can predict CSR names (e.g., generated from a deployment name), use resourceNames to restrict approval to specific CSRs:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: csr-approver-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: csr-approver
subjects:
- kind: ServiceAccount
  name: csr-approver
  namespace: ci-cd

Without resourceNames, the account can approve any CSR with the allowed verbs, which is broad but sometimes necessary if names are dynamic. An alternative is to use a validating webhook to enforce naming patterns.

Apply and verify:

kubectl apply -f rbac.yaml
kubectl auth can-i approve certificatesigningrequests --as=system:serviceaccount:ci-cd:csr-approver

Expected output: yes if permissions are correct.

Test the smallest change: create a test CSR, then approve it using the service account's credentials. Use kubectl --as to impersonate without creating a kubeconfig file:

kubectl create csr test-csr --signer kubernetes.io/kube-apiserver-client --requestor test
kubectl certificate approve test-csr --as=system:serviceaccount:ci-cd:csr-approver

If the approval succeeds, configuration is valid.

Keep sensitive data out of CI/CD: use short-lived tokens or workload identity instead of static kubeconfig files with long-lived credentials.

Verification and Diagnostics

After implementing automation, verify that CSRs are approved correctly and that certificates are issued.

Inspect a CSR before approval

kubectl get csr my-csr -o yaml

Example condensed output:

spec:
  signerName: kubernetes.io/kube-apiserver-client
  usages:
  - client auth
  request: LS0tLS1CRUdJTiBDRV...
status:
  conditions: []

Check the usages and signerName match expectations. A client auth CSR should not be approved for server auth.

Approve via kubectl

kubectl certificate approve my-csr

Expected output:

certificatesigningrequest.certificates.k8s.io/my-csr approved

Check status

kubectl get csr my-csr

If approved and signed, the CSR will show Approved,Issued under CONDITIONS.

Retrieve the issued certificate

kubectl get csr my-csr -o jsonpath='{.status.certificate}' | base64 -d > client.crt

Verify the certificate details:

openssl x509 -in client.crt -text -noout

Look for Subject, Issuer, Validity, and X509v3 Extended Key Usage to confirm the certificate matches the request.

Diagnose stuck CSRs

If a CSR remains Pending, check events:

kubectl describe csr my-csr

Common causes:

• No approver with permission to approve that signer
• The approver is not running (if using a custom controller)
• The signer's certificate/key misconfiguration

For CI/CD pipelines, log the CSR name and status after approval attempt:

STATUS=$(kubectl get csr my-csr -o jsonpath='{.status.conditions[?(@.type=="Approved")].status}')
if [ "$STATUS" != "True" ]; then
  echo "CSR not approved: $STATUS"
  exit 1
fi

This prevents pipeline continuation with an unsigned certificate.

Failure Modes and Recovery

Automated CSR approval can fail in several ways. Understanding each helps design recovery.

Failure 1: CSR is approved but certificate not issued

Symptom: CSR shows Approved but not Issued.

Cause: The signing controller (kube-controller-manager's CSR signer) is not running or its signing key is missing.

Diagnosis:

kubectl get pods -n kube-system | grep kube-controller-manager
kubectl logs -n kube-system kube-controller-manager-<node> | grep -i csr

Recovery: Ensure the controller manager has --cluster-signing-cert-file and --cluster-signing-key-file configured and the files exist. Restart the controller if needed.

Failure 2: Automated approver cannot approve CSR due to RBAC

Symptom: Pipeline logs show error: You must be logged in to the server (Unauthorized) or forbidden.

Cause: Missing RBAC rules or service account token expired.

Diagnosis:

kubectl auth can-i approve certificatesigningrequests --as=system:serviceaccount:ci-cd:csr-approver

Recovery: Review the ClusterRole and binding. Ensure the service account is in the correct namespace.

Failure 3: Wrong signer approved

Symptom: The issued certificate cannot be used for its intended purpose (e.g., client cert used as server cert).

Cause: Automation approved a CSR with a signer name that does not match the application's needs.

Recovery: Deny or delete the CSR and generate a new one with the correct signer:

kubectl delete csr bad-csr

Prevent by validating signer names in the automation script before approval.

Failure 4: Pipeline approves CSRs too broadly

Symptom: Unauthorized CSRs get approved automatically, potentially issuing credentials to attackers.

Cause: Overly permissive RBAC (no resourceNames, no signer filtering).

Recovery: Immediately revoke the binding, remove the service account token, and audit which CSRs were approved. Tighten RBAC to specific signer names and resource names.

Rollback for configuration changes: keep all YAML files in version control. If a change causes failures, revert to the previous commit and reapply.

Example rollback:

git revert <commit-hash>
kubectl apply -f rbac.yaml

Always verify the new state with kubectl auth can-i and a test CSR.

Operations Checklist

Use this checklist before and after deploying CSR automation to production.

Before deployment:

• [ ] Cluster version supported (v1.19+ for signer-specific approval)
• [ ] ServiceAccount created in a dedicated namespace
• [ ] ClusterRole scoped to certificatesigningrequests/approval and necessary read verbs
• [ ] ClusterRoleBinding ties the ServiceAccount to the ClusterRole
• [ ] If possible, resourceNames restrict approval to known CSR names
• [ ] Test with kubectl auth can-i as the service account
• [ ] Test with a sample CSR end-to-end in a staging environment
• [ ] Pipeline script logs CSR name and status
• [ ] Rollback plan documented (revert commit, reapply RBAC)

During operation:

• [ ] Monitor for unexpectedly approved CSRs (audit logs)
• [ ] Rotate service account tokens periodically (if using static tokens) or use workload identity
• [ ] Keep signing certificates valid and renew before expiry
• [ ] Regularly review RBAC permissions for least privilege

After failure recovery:

• [ ] Identify root cause and document
• [ ] Apply corrective action to automation
• [ ] Re-run test CSR to confirm fix
• [ ] Update operations runbook

Assign an owner: for example, the platform lead (e.g., Priya Shah) owns the CSR automation configuration and reviews it monthly or after any infrastructure change.

Common Pitfalls

Pitfall 1: Approving CSRs manually as a habit

Why it happens: Automation not set up or not trusted, so teams fall back to manual approval.

How to avoid: Implement CI/CD approval with logging and alerts. If manual approval remains, restrict it to break-glass accounts.

Pitfall 2: Using cluster-admin for the CI/CD service account

Why it happens: Quick and dirty setup, or lack of understanding of RBAC.

How to avoid: Use the scoped ClusterRole as shown above. Test with kubectl auth can-i --list to see all permissions.

Pitfall 3: Ignoring signer names

Why it happens: Older clusters defaulted to a single signer, so teams assume no distinction.

How to avoid: Always inspect spec.signerName and match approval logic to the expected signer.

Pitfall 4: Not verifying the issued certificate

Why it happens: Pipeline stops after approval, assuming issuance succeeded.

How to avoid: Check for Issued condition and decode the certificate to validate usages and expiry.

Pitfall 5: Storing long-lived kubeconfig in CI/CD

Why it happens: Simplicity.

How to avoid: Use short-lived tokens, OIDC, or cloud workload identity. Rotate secrets automatically.

Conclusion

Automating Kubernetes CSR approval in CI/CD is achievable with careful RBAC, clear verification steps, and robust failure handling. The key is to scope permissions tightly, validate signer names and usages, and ensure the signing controller is healthy.

Start with a low-risk implementation: create a test CSR, approve it via a restricted service account, and verify the certificate. Then integrate the approval step into your pipeline with appropriate logging and error handling.

By following the practices in this article, you can reduce manual toil, prevent misissued certificates, and maintain a secure and reliable certificate issuance process in your Kubernetes clusters.