Bash networking troubleshooting with practical examples: practical implementation guide

Intro

When a service cannot reach an endpoint, the cause is usually one of five things: name resolution, reachability, routing, filtering, or application-layer issues like TLS and proxy configuration. This guide shows how to diagnose each class of problem using Bash on Linux systems. It focuses on safe, verifiable steps you can run on development workstations and servers without guessing or making risky changes.

You will learn how to:

• Inventory your environment and network state up front
• Run fast, read-only checks that localize the fault
• Inspect DNS, ports, routes, firewalls, proxies, and TLS
• Interpret expected results and distinguish false positives
• Recover cleanly from small, temporary changes
• Create a repeatable operations checklist

All examples are constructed for illustration. Replace hostnames, domains, and addresses with values from your environment.

Version and Environment Inventory

Before changing anything, capture what you have. This speeds root cause analysis and creates a baseline for comparison.

Commands to snapshot system and tooling versions:

# OS and kernel
cat /etc/os-release
uname -a

# Bash and core network tools
bash --version | head -n1
ip -V || ip -V 2>/dev/null
ss -V 2>/dev/null || netstat -V 2>/dev/null
dig -v 2>/dev/null || host -v 2>/dev/null
command -v resolvectl >/dev/null && resolvectl --version

# Time sync can affect TLS validation
command -v timedatectl >/dev/null && timedatectl

Snapshot current network state (no changes made):

# Addresses, links, and routes
ip addr show
ip link show
ip route show

# Default route and routing decision for a target (constructed example)
ip route get 203.0.113.10

# DNS view
cat /etc/resolv.conf
command -v resolvectl >/dev/null && resolvectl status

# Name service switch order
grep -E '^[[:space:]]*hosts:' /etc/nsswitch.conf || echo 'nsswitch.conf hosts entry not found'

Collect service context:

# Local listeners (requires sudo for process names on some systems)
sudo ss -tulpn

# Environment proxies
env | grep -iE '^(http|https|no)_proxy='

Store snapshots under a timestamped directory so you can diff later:

TS=$(date +%Y%m%d-%H%M%S); mkdir -p net-snap-$TS; cd net-snap-$TS
cp /etc/resolv.conf . 2>/dev/null || true
{ ip addr; ip route; ip link; } > ip-state.txt
{ ss -tulpn || netstat -tulpn; } > sockets.txt 2>/dev/null
{ resolvectl status || true; } > resolve-status.txt 2>/dev/null
{ env | grep -iE '^(http|https|no)_proxy=' || true; } > proxies.txt

Safe Configuration Path

Use a narrow, reversible, and observable approach:

• Scope: pick one failing hostname and one port to investigate, for example api.example.test on TCP 443 (constructed example). Do not change global settings until you can reproduce and localize the fault.
• Backups: before editing files like /etc/hosts or /etc/nsswitch.conf, create dated backups: sudo cp /etc/hosts /etc/hosts.bak.$(date +%s).
• Read-only first: prefer commands that observe state (ip, ss, dig, curl -v) over commands that change state.
• Temporary, minimal changes only when needed: if you must adjust a rule or file for testing, record the command to undo it immediately.
• Validate locally: prove each step worked by capturing output and comparing to your baseline.

This keeps troubleshooting measurable and easy to roll back if something goes wrong.

Baseline Connectivity Tests

These quick checks tell you where to focus next.

# 1) Can we reach the target IP at all? (constructed IP)
ping -c 2 203.0.113.10 || echo 'ICMP may be blocked; continue with TCP tests'

# 2) Can we resolve the hostname to an IP?
getent hosts api.example.test || true

# 3) Resolve via the configured resolver(s)
dig +timeout=2 +tries=1 api.example.test A

# 4) Force IPv6 and IPv4 to see family-specific issues
dig +timeout=2 +tries=1 api.example.test AAAA

# 5) If you have an expected IP, test TCP reachability (constructed IP)
nc -vz 203.0.113.10 443

# 6) Test application layer quickly (no body download)
curl -I -sS https://api.example.test || curl -vkI https://api.example.test

Interpreting results:

• Name resolves but TCP fails: focus on routing, firewall, or target service availability.
• Name does not resolve: focus on DNS configuration and sources.
• IPv4 works but IPv6 fails (or vice versa): investigate dual-stack paths, AAAA records, and per-family firewalling.
• TCP handshake works but HTTP/TLS fails: inspect proxy env vars, SNI, and certificate chains.

Quick-reference: first-line tests

DNS Troubleshooting

Validate where your system gets answers and in what order.

1. Inspect resolution sources and order:

cat /etc/resolv.conf
grep -E '^[[:space:]]*hosts:' /etc/nsswitch.conf
command -v resolvectl >/dev/null && resolvectl status

• If /etc/resolv.conf points to 127.0.0.53, your system likely uses systemd-resolved. Use resolvectl to view actual upstream DNS servers.
• nsswitch.conf typically shows hosts: files dns. That means /etc/hosts is checked before DNS.

1. Compare answers from each source:

# NSS view
getent hosts api.example.test

# Resolver direct (current default)
dig +nocmd api.example.test A +noall +answer

# Query a specific DNS server (constructed IP)
dig @192.0.2.53 api.example.test A +noall +answer

# Check AAAA vs A asymmetry
dig api.example.test AAAA +short; dig api.example.test A +short

1. Follow CNAME chains and TTLs:

dig api.example.test +trace +nodnssec | sed -n '1,80p'

1. Flush caches (temporary, read-only effect):

# On systemd-resolved systems
sudo resolvectl flush-caches 2>/dev/null || sudo systemd-resolve --flush-caches 2>/dev/null || true

1. Temporarily pin a host for a test (constructed example):

# Add a single test line, then remove it after
sudo cp /etc/hosts /etc/hosts.bak.$(date +%s)
echo '203.0.113.10 api.example.test' | sudo tee -a /etc/hosts
getent hosts api.example.test
# Roll back when done: restore backup or remove the line

Common signals:

• SERVFAIL or REFUSED: upstream DNS issue or blocked query type.
• NXDOMAIN for A but valid AAAA (or the reverse): dual-stack mismatch.
• getent differs from dig: /etc/hosts or nsswitch.conf ordering is affecting results.

Name resolution sources and where to check

Ports and Services

Find out if something is listening locally, whether you can reach a remote port, and what the application says.

Local listeners:

# Note: requires sudo for process details on many systems
sudo ss -tulpn | sed -n '1,50p'

Key fields to read:

• Local Address: Port shows bound interfaces (0.0.0.0 means all IPv4, :: means all IPv6).
• State LISTEN confirms the server is awaiting connections.

Remote reachability and server banner:

# TCP handshake only (fast)
nc -vz api.example.test 443

# HTTP(S) headers without body
curl -I -sS https://api.example.test || curl -vkI https://api.example.test

# Show TLS handshake, SNI, and certificate chain
openssl s_client -connect api.example.test:443 -servername api.example.test -brief -showcerts </dev/null

Interpreting outcomes:

• Connection refused: target host is reachable but no process is listening on that port.
• Connection timed out: routing or firewall likely blocking; investigate hops and filtering.
• TLS: unknown CA or hostname mismatch indicates certificate or SNI issues.

Routing and MTU

Pinpoint the path and detect fragmentation problems that cause intermittent failures.

Which route is chosen for a target:

ip route get api.example.test

Trace the path and detect where packets stall:

# TCP-based traceroute often passes where ICMP is filtered
sudo traceroute -T -p 443 api.example.test 2>/dev/null || traceroute api.example.test

Detect MTU black holes (constructed payload sizes):

# Test with DF (do not fragment) and increasing sizes over IPv4
ping -M do -s 1472 -c 2 api.example.test  # 1472 + 28 IP/ICMP header ~= 1500 MTU
ping -M do -s 8972 -c 2 api.example.test  # For jumbo paths, if expected

Signals:

• Asymmetric routing: outgoing and incoming paths differ, often seen in traceroute vs server logs.
• PMTU issues: small requests work, large TLS handshakes stall; DF pings fail for sizes above a threshold.

Firewall Checks

Perform read-only inspections first. Avoid changing policies during diagnosis unless you have an agreed maintenance window and a rollback plan.

List rules safely across common stacks:

# nftables (modern)
sudo nft list ruleset

# iptables (legacy or compatibility layer)
sudo iptables -S; sudo iptables -L -n -v

# UFW (frontend to iptables)
sudo ufw status verbose

# firewalld (zones and services)
sudo firewall-cmd --get-active-zones
sudo firewall-cmd --zone=public --list-all

Review default policies and counters:

• DROP/REJECT policies with growing packet counters usually indicate filtering at that chain.
• DNAT/SNAT rules may redirect traffic unexpectedly; confirm destination after NAT.

If a temporary allow rule is necessary, write down the exact inverse command first. Example with iptables (constructed):

# Plan add and delete before running either:
# To add (example): sudo iptables -I INPUT -p tcp --dport 443 -s 198.51.100.0/24 -j ACCEPT
# To delete (rollback): sudo iptables -D INPUT -p tcp --dport 443 -s 198.51.100.0/24 -j ACCEPT

Firewall tooling map

Proxies and TLS

Silent proxies and certificate problems often masquerade as network outages.

Detect and interpret proxy variables:

env | grep -iE '^(http|https|no)_proxy='
# Example cleanup for testing (note: shell-only, not global)
unset http_proxy https_proxy no_proxy HTTP_PROXY HTTPS_PROXY NO_PROXY

Probe with and without proxy variables to confirm behavior changes. If a corporate proxy is required, test it explicitly:

HTTPS_PROXY=http://proxy.example.test:3128 curl -I -sS https://api.example.test

Check TLS handshakes and SNI:

openssl s_client -connect api.example.test:443 -servername api.example.test -brief -showcerts </dev/null

Look for:

• Server certificate subject and SANs include the requested hostname.
• Chain is complete up to a trusted root.
• System time is correct (TLS validation depends on accurate time).

Verification and Diagnostics

Define what success looks like and capture artifacts you can compare later.

Expected success signals by layer:

• DNS: getent hosts returns a stable set of IPs that match dig +short results for A/AAAA as appropriate.
• TCP: nc -vz host port reports succeeded consistently; no timeouts.
• HTTP: curl -I returns 2xx/3xx and completes in expected latency.
• TLS: openssl s_client shows a valid chain; no hostname mismatches; system time is correct.
• Routing: ip route get host shows the expected interface and source address; traceroute reaches the destination or last AS hop you control.
• Firewall: rule list and counters are consistent with allowed traffic; no unexpected drops on the relevant chains.

Automate and store a minimal verification bundle (constructed example target):

TARGET=api.example.test
PORT=443
OUT=verify-$(date +%Y%m%d-%H%M%S); mkdir -p "$OUT"
{
  echo '# DNS'; getent hosts "$TARGET"; dig +short "$TARGET" A; dig +short "$TARGET" AAAA
  echo '# TCP'; nc -vz "$TARGET" "$PORT" 2>&1 | sed 's/^/  /'
  echo '# HTTP'; curl -w '\n%{remote_ip} %{http_code} %{time_total}s\n' -I -sS https://"$TARGET" -o /dev/null
  echo '# ROUTE'; ip route get "$TARGET"
} | tee "$OUT/result.txt"

Review result.txt for consistent, expected outputs. Keep these bundles to compare across hosts or after changes.

Failure Modes and Recovery

Common pitfalls and how to fix or roll back safely.

1. Wrong /etc/hosts entry overrides DNS

• Symptom: getent hosts name returns an outdated IP that does not match dig.
• Fix: comment or remove the line. Recovery: restore the backup you made earlier: sudo cp /etc/hosts.bak.<timestamp> /etc/hosts.

1. Misordered nsswitch.conf causes slow or wrong lookups

• Symptom: delayed name resolution; unexpected results from getent.
• Fix: set hosts: files dns (typical). Recovery: restore from backup if changes do not help.

1. Stale DNS cache

• Symptom: dig to a specific upstream shows the new IP, but your system resolves the old IP.
• Fix: flush caches: sudo resolvectl flush-caches (if available) or restart the local resolver with care during a window. Recovery: revert service restart if it impacts other lookups.

1. IPv6-only or IPv6-preferred path breaks

• Symptom: curl hangs; dig AAAA returns an address; nc -6 -vz fails while -4 works.
• Fix: prefer IPv4 with curl -4 or adjust resolver order while you correct the IPv6 route/firewall. Recovery: remove temporary flags once IPv6 is fixed.

1. Firewall blocks new service port

• Symptom: nc times out; traceroute stops before target; counters increase on DROP rules.
• Fix (temporary test): add a specific ACCEPT rule limited by source and port; verify; then make a permanent rule via change control. Recovery: delete the test rule with the exact inverse command you planned.

1. Proxy intercept or wrong NO_PROXY

• Symptom: internal host only reachable without proxy; with proxy, TLS fails or wrong certificate is presented.
• Fix: set NO_PROXY=.example.test,10.0.0.0/8 appropriately. Recovery: unset or restore previous values.

1. MTU black hole

• Symptom: small pings work; HTTPS stalls during handshake; ping -M do -s 1472 fails.
• Fix: set a lower MTU on the affected interface temporarily for testing: sudo ip link set dev eth0 mtu 1400 (constructed interface); if it works, coordinate a permanent path MTU fix on the network. Recovery: revert: sudo ip link set dev eth0 mtu 1500 (original value).

1. Time skew breaks TLS

• Symptom: curl or openssl s_client reports certificate not yet valid or expired, but dates are fine elsewhere.
• Fix: correct system time and ensure time sync is enabled: timedatectl set-ntp true. Recovery: revert manual time edits if they cause drift; ensure NTP is stable.

Operations Checklist

Use this runbook during incidents. Replace placeholders with actual values.

1. Capture baseline

• cat /etc/os-release; uname -a; bash --version | head -n1
• ip addr; ip route; ip link; ss -tulpn (sudo where needed)
• cat /etc/resolv.conf; resolvectl status (if available)
• env | grep -iE '^(http|https|no)_proxy='

1. Localize the fault

• getent hosts <name>; dig +short <name> A and AAAA
• nc -vz <name or ip> <port>
• curl -I -sS https://<name> (add -k only to separate TLS from TCP failures)

1. DNS detail if needed

• Check /etc/hosts and /etc/nsswitch.conf
• dig @<dns_ip> <name> A +noall +answer
• resolvectl flush-caches (if safe)

1. Routing and path

• ip route get <name or ip>
• traceroute -T -p <port> <name>
• ping -M do -s 1472 <name> (adjust size as needed)

1. Firewall read-only check

• nft list ruleset or iptables -S
• ufw status verbose or firewall-cmd --list-all

1. Proxy and TLS

• Review proxy env; temporarily unset http_proxy https_proxy no_proxy
• openssl s_client -connect <name>:<port> -servername <name> -brief

1. Verify and record outcome

• Save key outputs; confirm expected signals by layer

1. Recovery if you changed anything

• Restore /etc/hosts or nsswitch.conf from backups
• Delete any temporary firewall rules
• Revert MTU or other interface changes
• Reapply correct proxy settings

Conclusion

Robust Bash networking troubleshooting starts with a disciplined inventory, then moves layer by layer: resolution, reachability, routing, filtering, and application specifics like proxies and TLS. By favoring read-only observations, making only minimal, reversible changes, and defining clear success signals, you reduce risk and shorten time to clarity.

Next steps you can take today:

• Turn the verification bundle into a small script that saves outputs to timestamped directories.
• Establish a narrow, measurable pilot on one service and host, then extend as patterns repeat.
• Keep a lightweight record of expected results (IPs, ports, TLS attributes) for your critical dependencies so you can spot drift immediately.

With these practices, you can diagnose most network issues quickly and confidently, and you will have the artifacts to prove each fix worked before you expand changes more broadly.