Kubernetes Ingress upgrade and migration with practical examples: practical implementation guide

Intro

Kubernetes Ingress upgrades touch one of the most sensitive edges of your platform: how users reach your apps. A good plan lets you move to supported APIs, modern controllers, and better traffic policies without breaking routes or TLS. This guide provides a practical, low-risk approach to upgrading and migrating Kubernetes Ingress, from inventory and planning through verification, failure modes, and rollback.

What you will get:

• Clear prerequisites and environment inventory steps.
• Three safe paths: API-only upgrade, in-place controller upgrade, and parallel migration.
• Concrete YAML, commands, and expected results.
• Failure modes, rollback, and a repeatable checklist.

The first pilot should be narrow, measurable, and easy to inspect locally and in staging before any production change. Keep scope tight, verify thoroughly, and then scale out.

Version and Environment Inventory

Before changing anything, capture a precise picture of what you have. This tells you which upgrade path is viable and prevents surprises.

1. Identify cluster and API capabilities

• Check cluster version:

kubectl version --short

• Confirm available Ingress API groups and preferred versions:

kubectl api-resources | grep -i ingress
kubectl explain ingress | head -n 5

Expected: networking.k8s.io/v1 is available on modern clusters (1.19+). If you still rely on extensions/v1beta1, plan an API migration first.

1. Find your Ingress controllers and classes

• List IngressClass resources:

kubectl get ingressclass

• Find controller deployments (commonly in a namespace like ingress-nginx, nginx-ingress, traefik, or vendor-specific):

kubectl get deploy, ds, svc -A | grep -i ingress

• Inspect the IngressClass for controller string (example for ingress-nginx):

kubectl get ingressclass -o yaml

Expected: You see something like spec.controller: k8s.io/ingress-nginx for the community ingress-nginx controller.

1. Map traffic and TLS

• List Ingresses and their addresses:

kubectl get ing -A -o wide

• Inventory hosts, paths, and TLS secrets (note: TLS secrets live in the same namespace as the Ingress):

kubectl get ing -A -o jsonpath='{range .items[*]}{.metadata.namespace} {.metadata.name} {.spec.rules[*].host} {.spec.tls[*].secretName}{"\n"}{end}'

• For each public Ingress, identify DNS records and current TTLs. Lower TTLs ahead of any cutover you plan.

1. Risk triage for incompatible features

• Note any annotations used for rewrites, timeouts, authentication, or canary. Some may behave differently between controller versions.
• Identify backends using named ports vs numeric ports, and services with nonstandard health checks.

Ingress API changes at a glance

Safe Configuration Path

Choose the smallest change that achieves your goal. Three proven options:

A) API-only upgrade (keep your controller)

• When: Your controller already supports networking.k8s.io/v1, and you only need to update resource manifests.
• Benefit: Minimal moving parts. Ideal for first pilot.

B) In-place controller upgrade (same class and Service)

• When: You want new controller features or CVE fixes but will keep the same LoadBalancer or NodePort Service.
• Benefit: No DNS change. Keep rollouts tight with surge and zero-downtime settings.

C) Parallel controller migration (new class and Service)

• When: You will replace the controller family or change major configuration. You run old and new controllers in parallel with different IngressClass values.
• Benefit: Safe canary or host-by-host cutover. Easy rollback by switching ingressClassName or DNS weight.

A) API-only upgrade example

Suppose you have this legacy Ingress (constructed example):

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: web-legacy
  namespace: prod
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  tls:
  - hosts: ["app.example.com"]
    secretName: tls-app
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: web
          servicePort: 80

Migrate to v1 with explicit pathType and structured backend:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web
  namespace: prod
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - app.example.com
    secretName: tls-app
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web
            port:
              number: 80

Apply safely:

kubectl apply -f web-ingress-v1.yaml
kubectl describe ing -n prod web

Expected: Address remains the same, TLS secret is found, and rules match your old Ingress.

B) In-place controller upgrade example (ingress-nginx)

1. Confirm existing class and Service

kubectl get ingressclass
kubectl get svc -n ingress-nginx

1. Stage a controlled deployment upgrade (constructed commands; adapt to your tooling)

• Ensure controller Deployment has a PodDisruptionBudget and readinessProbes are green.
• Set a safe rollout strategy (example):

kubectl patch deploy -n ingress-nginx ingress-nginx-controller \
  -p '{"spec":{"strategy":{"type":"RollingUpdate","rollingUpdate":{"maxUnavailable":0,"maxSurge":1}}}}'

1. Upgrade the controller image to the target version, then monitor:

kubectl set image deploy/ingress-nginx-controller -n ingress-nginx \
  controller=registry.k8s.io/ingress-nginx/controller: vX.Y.Z
kubectl rollout status deploy/ingress-nginx-controller -n ingress-nginx --timeout=3m
kubectl logs deploy/ingress-nginx-controller -n ingress-nginx --tail=100

Expected: No drop in endpoints, Service external IP remains, and existing Ingresses re-sync cleanly.

Rollback (if needed):

kubectl rollout undo deploy/ingress-nginx-controller -n ingress-nginx

C) Parallel controller migration example (class-based cutover)

Goal: Run a new controller in parallel, bind only selected Ingresses to it, verify, then move more traffic.

1. Create a new IngressClass for the new controller (constructed example):

apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: nginx-new
spec:
  controller: k8s.io/ingress-nginx

1. Deploy the new controller referencing this class (follow your vendor or chart docs). Ensure it creates a distinct Service such as ingress-nginx-new-controller with its own external IP.

1. Duplicate a single low-risk Ingress to target the new class:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-new
  namespace: prod
spec:
  ingressClassName: nginx-new
  tls:
  - hosts:
    - app-canary.example.com
    secretName: tls-app
  rules:
  - host: app-canary.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web
            port:
              number: 80

1. Create DNS for app-canary.example.com pointing to the new controller Service IP with a short TTL (constructed example: 60 seconds). Verify end-to-end before moving production hosts.

1. Host-by-host cutover: change the production Ingress to ingressClassName: nginx-new and re-apply. Alternatively, use weighted DNS to split traffic across the old and new controller Services during the transition.

Canary within a single controller (feature flag)

If you only need to canary a new backend version (not a new controller), you can use a canary Ingress with ingress-nginx.

Primary Ingress (stable):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-stable
  namespace: prod
spec:
  ingressClassName: nginx
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web
            port:
              number: 80

Canary Ingress (10% traffic to web-canary service):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-canary
  namespace: prod
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "10"
spec:
  ingressClassName: nginx
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-canary
            port:
              number: 80

Increase the weight as you gain confidence.

Verification and Diagnostics

Validate each stage with explicit checks and expected signals. Keep TTLs low and observe for a full error budget window before proceeding.

Core checks and expected signals

Additional diagnostics

• Confirm generated config (ingress-nginx):

# Identify a controller pod
POD=$(kubectl get pods -n ingress-nginx -l app.kubernetes.io/component=controller -o jsonpath='{.items[0].metadata.name}')
# Inspect a snippet of the generated NGINX config
kubectl exec -n ingress-nginx $POD -- cat /etc/nginx/nginx.conf | head -n 50

• Validate TLS secret content and namespace alignment:

kubectl get secret tls-app -n prod -o yaml | grep -E 'tls\.crt|tls\.key'

• Confirm Service endpoints:

kubectl get endpoints web -n prod -o yaml | grep addresses -A2

• Load test a small sample (constructed example):

for i in $(seq 1 50); do curl -s -o /dev/null -w "%{http_code} %{time_total}\n" https://app.example.com/healthz; done | sort | uniq -c

Expected: Stable 200s with consistent timings.

Failure Modes and Recovery

Plan for failure so you can recover quickly and confidently.

1. 404 Not Found after cutover

• Cause: Wrong host, path mismatch, or missing default backend. In v1, pathType is required and changes matching behavior.
• Fix: Ensure pathType: Prefix or Exact as intended, confirm host spelling, and that the backend service exists.
• Rollback: Re-apply the last known-good Ingress manifest (keep it on hand), or switch ingressClassName back to the old class if you used parallel migration.

1. 502/504 from upstream

• Cause: Service endpoints not ready, port mismatch, or readiness checks failing.
• Fix: Verify service selectors, endpoints, and pod readiness. Confirm backend port number vs name.
• Rollback: Scale canary weight to 0 or revert Ingress to stable backend.

1. TLS handshake or certificate errors

• Cause: TLS secret not found, wrong namespace, or missing SAN for the host.
• Fix: Ensure the secret exists in the Ingress namespace and contains valid tls.crt and tls.key for the host.
• Rollback: Reinstate the previous TLS secret and re-apply.

1. Controller never assigns an address

• Cause: LoadBalancer provisioning delay, cloud permissions, or misconfiguration.
• Fix: Inspect Service events, cloud provider logs, and controller logs. Verify Service type and annotations.
• Rollback: Keep old controller as the primary; avoid DNS change until the new Service has an address.

1. Annotation behavior change

• Cause: Some annotations may differ between controller versions.
• Fix: Compare supported annotations, test on a canary host, and remove or adjust deprecated keys.
• Rollback: Restore the previous annotation set and controller version.

1. Unexpected routing after API migration

• Cause: Assumptions from v1beta1 path matching no longer hold.
• Fix: Set pathType explicitly; for regex or rewrites, ensure controller-specific annotations are correct.
• Rollback: If supported by your cluster, revert to the previous manifest while you correct paths.

Rollback playbooks

A) API-only upgrade rollback

# Re-apply the previously working manifest
kubectl apply -f web-ingress-previous.yaml
# Confirm status
kubectl describe ing -n prod web

B) In-place controller upgrade rollback

# Roll back the deployment to the prior ReplicaSet
kubectl rollout undo deploy/ingress-nginx-controller -n ingress-nginx
# Watch pods return to Ready
kubectl rollout status deploy/ingress-nginx-controller -n ingress-nginx --timeout=3m

C) Parallel controller migration rollback

• Repoint production Ingress from ingressClassName: nginx-new back to ingressClassName: nginx and apply.
• If you used DNS split, set weight back to 100% old controller and 0% new controller.
• Remove the canary or duplicate Ingresses targeting the new class once stable.

Verification after rollback:

kubectl get ing -A -o wide
kubectl logs deploy/ingress-nginx-controller -n ingress-nginx --tail=100
curl -I https://app.example.com/healthz

Expected: Previous behavior restored with healthy responses.

Operations Checklist

Use this checklist to guide each upgrade or migration. Mark items done; do not skip verification gates.

Planning and inventory

• [ ] Record cluster version and confirm networking.k8s.io/v1 availability.
• [ ] List all Ingresses, hosts, paths, and TLS secrets.
• [ ] Identify current Ingress controllers, classes, and Services.
• [ ] Lower DNS TTLs for affected hosts.

Pilot selection

• [ ] Choose a single low-risk host or path as the first pilot.
• [ ] Decide on approach: API-only, in-place upgrade, or parallel migration.
• [ ] Prepare manifests for the pilot, including pathType and ingressClassName.

Change execution

• [ ] Apply changes to a non-production environment first; verify end-to-end.
• [ ] For in-place controller upgrades, ensure surge/zero-unavailable rollout.
• [ ] For parallel migration, create a new IngressClass and new controller Service.
• [ ] If canarying, start at 5-10% weight and observe.

Verification

• [ ] Check Ingress status and ADDRESS fields.
• [ ] Review controller logs for errors and config reloads.
• [ ] Validate TLS certs and SANs for all hosts.
• [ ] Probe health endpoints; confirm latency and error rates.

Decision and scale-out

• [ ] Hold steady state for an agreed observation window.
• [ ] Increase canary weight or move additional hosts.
• [ ] Document any annotation or behavior changes discovered.

Rollback readiness

• [ ] Keep prior manifests and controller versions noted.
• [ ] Have DNS weights or ingressClassName toggles prepared.
• [ ] Verify rollback commands in a staging environment.

Conclusion

Upgrading and migrating Kubernetes Ingress safely is about scoping the change, verifying at each step, and keeping a clean path to rollback. Start with an API-only upgrade where possible, then proceed to in-place controller upgrades or parallel migrations when you need new features or a different controller family. Keep pilots small, use explicit pathType and ingressClassName, validate with logs and probes, and store known-good manifests for instant recovery. With a short, repeatable checklist and a narrow first pilot, you can modernize Ingress with confidence and minimal disruption.