## Intro Ingress exposes applications to the outside world. A small misconfiguration can leak headers, accept weak ciphers, bypass authentication, or overexpose network paths. This guide shows how to harden Kubernetes Ingress with practical, verifiable steps and clear rollback procedures. You will: - Inventory versions and exposure points so changes are compatible and testable. - Apply a safe, scoped configuration path for TLS, mTLS, IP allow-lists, rate limiting, secrets, RBAC, NetworkPolicy, and load balancer controls. - Verify protections with concrete commands and expected results. - Recognize failure modes and recover quickly. - Keep an operational checklist you can run regularly. Constructed examples are labeled as such. Adjust names, namespaces, and limits to your environment. ## Version and Environment Inventory Before changing anything, record what you are hardening. Mismatched versions or ambiguous topologies cause most surprises. Prerequisites: - You have cluster admin or delegated rights to create/modify Ingress, Secrets, RBAC, and NetworkPolicy in the target namespaces. - You know which Ingress controller you use (for example, NGINX Ingress Controller) and its namespace. - You control or can update DNS for your hostnames. - You have or can issue valid TLS certificates for your domains. Constructed example inventory:
ItemExample valueWhy it matters
Kubernetes version1.28 (constructed example)Determines API availability (e.g., ValidatingAdmissionPolicy).
Ingress controlleringress-nginx 1.9.x (constructed example)Defines supported annotations and ConfigMap keys.
IngressClass namepublic-nginx (constructed example)Explicit binding prevents accidental default routing.
DNS zoneexample.test (constructed example)Controls host routing and TLS SANs.
Load balancerCloud L4 with source ranges (constructed example)You can restrict inbound IPs at the LB edge.
TLS sourceInternal CA and public CA (constructed example)Choose mTLS roots and server certs.
Commands to capture current state: kubectl get ns kubectl get deploy,ds,svc,cm -n ingress-nginx kubectl get ingressclass kubectl get ingress -A -o wide kubectl version --short ## Safe Configuration Path Start with a narrow, high-value target that you can test and roll back quickly (for example, one hostname). Then expand. ### 1) Bind to an explicit IngressClass Ensure every Ingress resource targets a known controller. This prevents unknown defaults from claiming traffic. Constructed example IngressClass: apiVersion: networking.k8s.io/v1 kind: IngressClass metadata: name: public-nginx spec: controller: k8s.io/ingress-nginx ### 2) Enforce TLS and secure defaults Use a Kubernetes TLS secret, redirect HTTP to HTTPS, and enable strict transport security (HSTS). The following example demonstrates secure annotations for NGINX Ingress Controller. Create a TLS secret (constructed example): kubectl create secret tls web-tls \ --namespace app \ --key server.key \ --cert server.crt Apply an Ingress with secure annotations (constructed example): apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: web namespace: app annotations: nginx.ingress.kubernetes.io/ssl-redirect: "true" nginx.ingress.kubernetes.io/ssl-protocols: "TLSv1.2 TLSv1.3" nginx.ingress.kubernetes.io/hsts: "true" nginx.ingress.kubernetes.io/hsts-max-age: "63072000" nginx.ingress.kubernetes.io/hsts-include-subdomains: "true" nginx.ingress.kubernetes.io/hsts-preload: "true" nginx.ingress.kubernetes.io/server-tokens: "false" nginx.ingress.kubernetes.io/proxy-body-size: "1m" # Optional: fine-tune ciphers if you manage a controlled client set # nginx.ingress.kubernetes.io/ssl-ciphers: "ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256" labels: security-tier: hardened spec: ingressClassName: public-nginx tls: - hosts: - app.example.test secretName: web-tls rules: - host: app.example.test http: paths: - path: / pathType: Prefix backend: service: name: web-svc port: number: 80 Controller-wide defaults can be pinned in the controller ConfigMap (constructed example): apiVersion: v1 kind: ConfigMap metadata: name: ingress-nginx-controller namespace: ingress-nginx data: server-tokens: "false" ssl-protocols: "TLSv1.2 TLSv1.3" # Keep ciphers concise; avoid obsolete suites # ssl-ciphers: "TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256" ### 3) Add client certificate authentication (mTLS) when appropriate For admin portals or internal APIs, require client certificates to reach the app. Store your CA bundle in a secret. Example (constructed): kubectl -n app create secret generic client-ca --from-file=ca.crt=ca.pem apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: admin namespace: app annotations: nginx.ingress.kubernetes.io/ssl-redirect: "true" nginx.ingress.kubernetes.io/auth-tls-secret: "app/client-ca" nginx.ingress.kubernetes.io/auth-tls-verify-client: "on" nginx.ingress.kubernetes.io/auth-tls-verify-depth: "1" # Optionally pass client cert details to upstream nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream: "true" nginx.ingress.kubernetes.io/whitelist-source-range: "203.0.113.0/24,198.51.100.0/24" labels: security-tier: hardened spec: ingressClassName: public-nginx tls: - hosts: ["admin.example.test" ] secretName: web-tls rules: - host: admin.example.test http: paths: - path: / pathType: Prefix backend: service: name: admin-svc port: number: 8443 ### 4) Add IP allow-lists and rate limiting Constrain traffic sources and limit abuse. Constructed example annotations: metadata: annotations: nginx.ingress.kubernetes.io/whitelist-source-range: "203.0.113.0/24,198.51.100.0/24" nginx.ingress.kubernetes.io/limit-rps: "10" nginx.ingress.kubernetes.io/limit-burst: "20" Set the ingress controller Service to preserve client IPs so rate limiting and allow-lists evaluate real addresses: apiVersion: v1 kind: Service metadata: name: ingress-nginx-controller namespace: ingress-nginx spec: type: LoadBalancer externalTrafficPolicy: Local loadBalancerSourceRanges: - 0.0.0.0/0 # tighten for admin endpoints, e.g., to corporate CIDRs ### 5) Secure Secrets and limit permissions - Keep TLS secrets in the same namespace as the Ingress that consumes them. - Restrict who can create or modify Ingress and Secrets. Constructed example RBAC limiting Ingress changes to a specific group in the app namespace: apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: ingress-maintainer namespace: app rules: - apiGroups: ["networking.k8s.io" ] resources: ["ingresses" ] verbs: ["get","list","watch","create","update","patch" ] - apiGroups: ["" ] resources: ["secrets" ] resourceNames: ["web-tls", "client-ca" ] verbs: ["get","list","watch","update","patch" ] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: ingress-maintainer-binding namespace: app subjects: - kind: Group name: app-team # constructed example identity roleRef: kind: Role name: ingress-maintainer apiGroup: rbac.authorization.k8s.io Admission control can enforce standards cluster-wide. Constructed example requiring TLS and a specific IngressClass: apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicy metadata: name: enforce-ingress-tls-and-class spec: matchConstraints: resourceRules: - apiGroups: ["networking.k8s.io" ] apiVersions: ["v1" ] operations: ["CREATE","UPDATE" ] resources: ["ingresses" ] validations: - expression: "has(object.spec.tls) && size(object.spec.tls) > 0" message: "Ingress must configure TLS." - expression: "object.spec.ingressClassName == 'public-nginx'" message: "Ingress must target approved ingressClassName." --- apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingAdmissionPolicyBinding metadata: name: enforce-ingress-tls-and-class-binding spec: policyName: enforce-ingress-tls-and-class validationActions: ["Deny" ] ### 6) Constrain network paths Ensure the controller can reach only the backends it needs, and peers cannot reach the controller unexpectedly. Constructed example NetworkPolicy allowing only the ingress controller to call your service: apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: allow-only-ingress namespace: app spec: podSelector: matchLabels: app: web policyTypes: ["Ingress" ] ingress: - from: - namespaceSelector: matchLabels: kubernetes.io/metadata.name: ingress-nginx podSelector: matchLabels: app.kubernetes.io/name: ingress-nginx ports: - protocol: TCP port: 80 ## Verification and Diagnostics Use the following checks after each change. Verify on a single hostname first. Quick reference of verification commands and expected signals (constructed results):
CheckCommandExpected
Ingress is admittedkubectl describe ingress -n app webShows IngressClass public-nginx; rules and tls populated.
Address allocatedkubectl get ingress -n app web -o wideADDRESS populated with LB IP/hostname.
HTTP redirects to HTTPScurl -I http://app.example.test301/308 with Location: https://app.example.test/
TLS min versionopenssl s_client -connect app.example.test:443 -tls1_0Handshake fails; verify with TLSv1.2 succeeds.
HSTS presentcurl -I https://app.example.testStrict-Transport-Security header present.
mTLS enforcementcurl -I https://admin.example.test400/401/403 without client cert; 200 with client cert.
Rate limitingfor i in {1..50}; do curl -s -o /dev/null -w "%{http_code}\n" https://app.example.test; doneSome 429 responses under tight RPS.
Detailed steps: - Admission and class binding kubectl describe ingress -n app web Expected: Annotations shown, IngressClassName: public-nginx, Events have no Deny messages from the ValidatingAdmissionPolicy. - TLS redirect and headers curl -I http://app.example.test curl -I https://app.example.test Expected: HTTP returns 301/308 to https. HTTPS returns 200 and includes Strict-Transport-Security. No Server: nginx version leaks when server-tokens is false. - Protocol floor (TLSv1.2+) # Expect failure with obsolete protocols openssl s_client -connect app.example.test:443 -tls1_0 /dev/null && curl -Is http://web.app.svc.cluster.local:80 || true" Expected: Connection timeout or refusal when the policy blocks traffic. ## Failure Modes and Recovery Things that commonly break and how to recover quickly. - TLS secret issues - Symptom: 526/525 style TLS failures at the edge, or controller logs show no valid certificate. - Check: The secret type must be kubernetes.io/tls and match the Ingress tls.secretName. - Fix: kubectl -n app get secret web-tls -o yaml | grep type kubectl -n app delete secret web-tls kubectl -n app create secret tls web-tls --key server.key --cert server.crt kubectl -n app rollout restart deploy/ingress-nginx-controller # if needed - HTTP not redirecting to HTTPS - Symptom: HTTP 200 on port 80. - Check: ssl-redirect annotation is set to "true" at the Ingress or controller level. - Fix: Add the annotation and reapply. Verify with curl -I. - TLS 1.0/1.1 still accepted - Symptom: openssl -tls1_0 succeeds. - Check: ssl-protocols in annotations/ConfigMap. - Fix: Set ssl-protocols to "TLSv1.2 TLSv1.3\); restart controller if required. - mTLS locks out valid users - Symptom: Everyone gets 400/401/403. - Checks: CA bundle secret name and file key, verify-depth, and whether clients send a cert. - Quick rollback: Remove auth-tls-* annotations from the Ingress and reapply. Keep a backup manifest to restore fast. - Aggressive rate limiting - Symptom: Many 429 responses during normal load. - Fix: Increase limit-rps and limit-burst or remove temporarily. Verify with burst test loop. - IP allow-list too tight - Symptom: 403 for expected users. - Fix: Add CIDR ranges and reapply. Prefer testing allow-list changes on a staging host before production. - NetworkPolicy blocks backends - Symptom: 502/504 from the controller; controller logs show upstream connect failures. - Checks: Namespace and pod label selectors in the policy, target port numbers. - Fix: Adjust selectors or ports; re-test connectivity from the controller namespace. - Load balancer source ranges block health checks - Symptom: LB never reports healthy; Ingress ADDRESS stays pending or traffic blackholes. - Fix: Include the provider health probe IP ranges in loadBalancerSourceRanges. Confirm with provider docs. As a quick test, temporarily widen to 0.0.0.0/0 to confirm the diagnosis, then re-tighten. Rollback guidance: - Always export the current manifest before changes: kubectl -n app get ingress web -o yaml > web-backup-$(date +%Y%m%d%H%M%S).yaml - If a change breaks traffic, immediately reapply the backup: kubectl apply -f web-backup-20240101123456.yaml - For controller-wide changes (ConfigMap, Service), stage updates in a non-production cluster first. If production is impacted, revert the ConfigMap/Service to the previous version and restart the controller deployment to pick up reverted settings. Certificate expiry check and recovery (constructed example): # Decode and check the Not After date kubectl -n app get secret web-tls -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -enddate # Renew by replacing the secret, then re-verify kubectl -n app create secret tls web-tls --key new.key --cert new.crt --dry-run=client -o yaml | kubectl apply -f - ## Operations Checklist Use this list during regular reviews and before/after changes. - Inventory - Confirm Kubernetes, controller, and IngressClass versions. - List all Internet-exposed hostnames and their namespaces. - TLS and headers - All public hosts enforce HTTPS with HSTS; TLSv1.2+ only. - Secrets are of type kubernetes.io/tls and not shared across unrelated namespaces. - No server version leaks (server-tokens false). - Access control - mTLS required for admin or internal endpoints. - IP allow-lists present for restricted services. - Rate limits tuned to normal traffic with headroom. - Permissions and policy - RBAC grants only specific teams the right to modify Ingress and related secrets. - ValidatingAdmissionPolicy enforces TLS and approved IngressClass. - NetworkPolicies allow only the controller to reach backend services. - Load balancer - externalTrafficPolicy=Local when you need client IPs. - loadBalancerSourceRanges restricts inbound IPs where feasible. - Verification - curl and openssl checks pass for each host. - Controller logs show no repeated TLS or upstream errors. - Recovery readiness - Manifests exported before changes; known-good backups exist. - Rollback steps are documented and tested on a staging host. ## Conclusion You hardened Kubernetes Ingress by explicitly binding to an IngressClass, enforcing TLS and HSTS, adding mTLS where needed, constraining traffic with IP allow-lists and rate limits, locking down secrets and RBAC, applying NetworkPolicy, and tightening load balancer exposure. You verified each control with observable checks and prepared rollback procedures for fast recovery. Next, expand from one hardened hostname to all Internet-facing services, keeping changes narrow, testable, and reversible as you go. By iterating with a clear plan and measurable checks, you reduce rework and keep your entry points secure.