## Intro TLS certificates are the foundation of secure communication on the internet. When they fail, applications break, users see warnings, and production incidents occur. Troubleshooting TLS certificate problems often involves multiple layers: certificate validity, chain of trust, hostname matching, protocol versions, cipher suites, and configuration in web servers or reverse proxies. This guide provides a practical, step-by-step approach for developers and DevOps engineers to diagnose and resolve common TLS certificate issues. It includes concrete commands, configuration snippets, and expected outputs, covering prerequisites, safe implementation, verification, failure modes, recovery, and an operations checklist. ## Version and Environment Inventory Before troubleshooting, establish the exact environment. Determine the operating system, web server or proxy software, TLS library versions, and certificate details. This information helps narrow down potential causes and ensures consistent reproduction. Example commands to gather environment information: # Check OS version cat /etc/os-release # Check Nginx version and TLS support nginx -V 2>&1 | grep -o 'with-http_ssl_module' nginx -V 2>&1 | grep -o 'built with OpenSSL [^ ]*' # Check OpenSSL version openssl version -a # Check certificate details (assuming you have the certificate file) sudo openssl x509 -in /etc/nginx/ssl/example.com.crt -noout -subject -issuer -dates Expected output for certificate details: subject= /CN=example.com issuer= /C=US/O=Let's Encrypt/CN=R3 dates: notBefore=Jan 1 00:00:00 2024 GMT notAfter=Mar 31 23:59:59 2025 GMT For situations involving a reverse proxy like Nginx, note the upstream backend and whether TLS termination occurs at the proxy or backend. This affects where certificate checks are performed. Document the topology, including load balancers, CDN, and internal services. ## Safe Configuration Path When modifying TLS configuration, always work on a controlled path: test changes in a staging environment or local instance first, then apply to production with rollback options. For Nginx, use nginx -t to test configuration syntax before reloading. Example safe configuration workflow: - Backup existing configuration: sudo cp /etc/nginx/sites-available/example.com.conf /etc/nginx/sites-available/example.com.conf.bak - Make scoped changes (e.g., update certificate paths or protocols): server { listen 443 ssl; server_name example.com; ssl_certificate /etc/nginx/ssl/example.com.crt; ssl_certificate_key /etc/nginx/ssl/example.com.key; ssl_protocols TLSv1.2 TLSv1.3; ssl_ciphers HIGH:!aNULL:!MD5; } - Test configuration: sudo nginx -t Expected output if successful: nginx: configuration file /etc/nginx/nginx.conf test is successful - Reload Nginx gracefully: sudo systemctl reload nginx - Verify the change with a local request before trusting production: curl -vI --resolve example.com:443:127.0.0.1 https://example.com This ensures the change is limited in scope and reduces risk. ## Verification and Diagnostics Use a combination of command-line tools to verify certificates, chain of trust, and protocol handshakes. Key tools: openssl , curl , echo | openssl s_client , and web server logs. ### Certificate Chain Verification openssl verify -CAfile /etc/ssl/certs/ca-certificates.crt /etc/nginx/ssl/example.com.crt Expected output if valid: /etc/nginx/ssl/example.com.crt: OK If chain is broken, output may show: error 20 at 0 depth lookup: unable to get local issuer certificate ### Remote Server Certificate Inspection echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null | openssl x509 -noout -text Check for Subject Alternative Name (SAN) includes the hostname. ### Testing with curl Verbose output reveals TLS errors: curl -v https://example.com 2>&1 | grep -E 'SSL|TLS|certificate|handshake' Common error messages: - SSL certificate problem: unable to get local issuer certificate (missing intermediate) - SSL: no alternative certificate subject name matches target host name (hostname mismatch) ### Checking Expiration openssl x509 -enddate -noout -in /etc/nginx/ssl/example.com.crt Expected output: notAfter=Mar 31 23:59:59 2025 GMT ### Log Analysis Nginx logs may show TLS errors. Example /var/log/nginx/error.log : SSL_do_handshake() failed (SSL: error:14094416:SSL routines: ssl3_read_bytes: sslv3 alert certificate unknown: SSL alert number 46) This indicates the client rejected the server certificate, often due to missing intermediate or trust issues. ## Failure Modes and Recovery Common failure modes include expired certificates, hostname mismatch, weak protocols, untrusted issuer, and misconfigured chain. Each requires a specific recovery approach. ### Expired Certificate Renew immediately. For Let's Encrypt, use certbot: sudo certbot renew --dry-run sudo certbot renew Then reload web server. Verify new expiration date. ### Hostname Mismatch Ensure the certificate's SAN includes the exact hostname (including wildcards if applicable). If not, obtain a new certificate. ### Missing Intermediate Concatenate the intermediate certificate with the server certificate in the correct order: cat example.com.crt intermediate.crt > fullchain.crt Update Nginx config to use fullchain.crt as ssl_certificate . ### Protocol or Cipher Issues Adjust ssl_protocols and ssl_ciphers to support modern clients (e.g., enable TLSv1.2/1.3). Test with: nmap --script ssl-enum-ciphers -p 443 example.com or use testssl.sh . ### Recovery via Rollback Always keep a backup of the previous working configuration and certificates. If a change causes failure, revert: sudo cp /etc/nginx/sites-available/example.com.conf.bak /etc/nginx/sites-available/example.com.conf sudo systemctl reload nginx ### Automated Monitoring Set up checks to alert on expiration. Example cron entry: 0 9 * * * /usr/local/bin/check_cert.sh example.com Where check_cert.sh exits non-zero if certificate expires within 14 days. ## Common Pitfalls and How to Avoid Them Many TLS failures stem from recurring mistakes. Here are the most frequent pitfalls, why they happen, and how to avoid or recover from them. ### 1. Forgetting to include intermediate certificates Why it happens : The server certificate alone is not trusted by clients unless they can build a chain to a root CA. If the intermediate is missing, the client cannot verify the chain. How to avoid : Always install the full chain (server certificate plus intermediates) as provided by the CA. For Let's Encrypt, use fullchain.pem . How to recover : Combine the server and intermediate certificates in order: cat server.crt intermediate.crt > fullchain.crt Then configure your web server to use fullchain.crt . ### 2. Not checking the certificate expiry date Why it happens : Certificates expire silently. Without monitoring, a certificate can expire and cause an outage. How to avoid : Set up automated monitoring with tools like certbot renew --dry-run in cron, or use an external monitoring service that checks expiration. How to recover : Renew the certificate immediately, then reload the web server. ### 3. Hostname mismatch due to missing SAN Why it happens : Older certificates may use only the Common Name (CN) field. Modern browsers require the hostname to be in the Subject Alternative Name (SAN) extension. If the SAN does not include the exact hostname, the browser rejects the certificate. How to avoid : When generating a CSR, include all needed SANs (e.g., example.com, www.example.com). How to recover : Obtain a new certificate with the correct SANs. ### 4. Using outdated TLS protocols or weak ciphers Why it happens : Older configurations may still enable TLSv1.0 or SSLv3 for compatibility, but these are now considered insecure and may be blocked by browsers or security scans. How to avoid : Disable outdated protocols and weak ciphers. Use only TLSv1.2 and TLSv1.3, and prefer strong cipher suites. How to recover : Update your web server configuration: ssl_protocols TLSv1.2 TLSv1.3; ssl_ciphers HIGH:!aNULL:!MD5; Then test with nmap --script ssl-enum-ciphers or testssl.sh . ### 5. Misconfiguration in reverse proxy setups Why it happens : In complex topologies with load balancers and reverse proxies, certificates may be installed on the proxy but not on the backend, or vice versa. Clients may see errors if the proxy forwards TLS incorrectly. How to avoid : Clearly document where TLS termination occurs. Verify each hop with openssl s_client . How to recover : Trace the request path and ensure certificates are correctly installed at the termination point. ## Operations Checklist Use this checklist for regular TLS operations and troubleshooting reviews. Assign a single owner for each item and review at least monthly.
CheckCommand / MethodExpected ResultOwnerReview Frequency
Certificate expirationopenssl x509 -enddate -noout -in cert.pemDate in the future (>= 14 days recommended)DevOps EngineerWeekly
Certificate chain validityopenssl verify -CAfile ca.pem cert.pemOKSecurity LeadMonthly
Hostname matchCompare request hostname with SANSAN includes hostnameDevOps EngineerWeekly
Protocol supportnmap --script ssl-enum-ciphers -p 443 hostNo weak protocols (SSLv2/3, TLSv1.0)Security LeadMonthly
Web server config testnginx -t (or equivalent)Syntax OKDevOps EngineerEvery change
Backup existenceCheck backup file timestampRecent backup existsDevOps EngineerMonthly
Monitoring alertsReview alert configurationAlert thresholds set for expiration and misconfigurationSRE LeadQuarterly
Regularly review logs for TLS errors and adjust configurations as needed. ## Conclusion TLS certificate troubleshooting requires a systematic approach: gather environment details, safely modify configuration, verify with diagnostic tools, and plan recovery for common failures. By using the commands and procedures outlined, practitioners can quickly identify and resolve issues such as expired certificates, chain problems, and hostname mismatches. Implement the operations checklist to maintain certificate health and prevent future incidents. Start with a narrow, measurable pilot change to validate your process before broader deployment.