Bash CI/CD Automation: A Production-Ready Guide with Practical Examples

Intro

Bash is still the glue of many CI/CD pipelines. It is available everywhere, fast to iterate, and ideal for orchestrating tools like Git, Docker, and Kubernetes. This guide shows a production-ready approach to Bash in CI/CD: strict shell safety, clean logging, modular functions, retries and timeouts, validation before deploy, health verification, and fast rollbacks. You will also see GitHub Actions and GitLab CI examples you can copy into your repos today.

What you will build:

• A reusable Bash library (strict modes, logging, traps, retries, timeouts, locks)
• Scripts for lint, test, build, validate, deploy, health-check, and rollback
• CI pipelines for GitHub Actions and GitLab CI
• Practical rollback patterns for Kubernetes and Docker Compose

Recommended repo layout:

.
├─ scripts/
│  ├─ lib.sh
│  ├─ lint.sh
│  ├─ test.sh
│  ├─ build.sh
│  ├─ validate.sh
│  ├─ deploy.sh
│  ├─ healthcheck.sh
│  └─ rollback.sh
├─ docker/
│  └─ compose.yaml
├─ k8s/
│  ├─ deployment.yaml
│  └─ service.yaml
└─ app/
   └─ ...

1) Safe Bash foundation (scripts/lib.sh)

Use strict modes, predictable word splitting, defensive error handling, and structured logging.

#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'

# Colors only if stdout is a TTY
if [ -t 1 ]; then
  GREEN='\033[0;32m'; YELLOW='\033[0;33m'; RED='\033[0;31m'; NC='\033[0m'
else
  GREEN=''; YELLOW=''; RED=''; NC=''
fi

log()  { printf "%sINFO%s  %s\n"  "$GREEN" "$NC" "$*"; }
warn() { printf "%sWARN%s  %s\n"  "$YELLOW" "$NC" "$*"; }
err()  { printf "%sERROR%s %s\n" "$RED" "$NC" "$*" 1>&2; }
fail(){ err "$*"; exit 1; }

require() { command -v "$1" >/dev/null 2>&1 || fail "Missing command: $1"; }
assert_env(){ for v in "$@"; do [ -n "${!v-}" ] || fail "Missing env var: $v"; done; }

# Basic error context
_on_error() {
  local ec=$? line=${1:-}
  err "Command failed at line $line (exit=$ec). Enable trace with: export TRACE=1"
}

# Optional tracing for debugging
[ "${TRACE-}" = "1" ] && set -x
trap '_on_error $LINENO' ERR

# Run CMD with retries
with_retry() {
  local attempts=${1:-5} sleep_s=${2:-2}; shift 2
  local n=1
  until "$@"; do
    if [ $n -ge "$attempts" ]; then
      err "Retry failed after $attempts attempts: $*"; return 1
    fi
    warn "Attempt $n failed. Retrying in ${sleep_s}s..."; sleep "$sleep_s"; n=$((n+1))
  done
}

# Timebox a command if 'timeout' exists
timebox() {
  local seconds=$1; shift
  if command -v timeout >/dev/null 2>&1; then
    timeout "$seconds" "$@"
  else
    warn "timeout not found; running without time limit"
    "$@"
  fi
}

# Serialize critical sections
lock_run() {
  local lockfile=$1; shift
  require flock
  exec 9>"$lockfile"
  flock -n 9 || fail "Another process holds $lockfile"
  "$@"
}

# Simple on-exit cleanup hook
_cleanup_funcs=()
on_exit() { _cleanup_funcs+=("$*"); }
trap 'for f in "${_cleanup_funcs[@]}"; do eval "$f" || true; done' EXIT

Key points:

• set -euo pipefail and IFS reduce footguns and hidden failures.
• trap with a custom handler gives actionable error context.
• with_retry and timebox protect flaky or long-running operations.
• lock_run serializes deploys to avoid race conditions.

2) Lint and test scripts

Lint your Bash and run fast unit checks. Replace the test snippet with your language’s test runner if needed.

scripts/lint.sh

#!/usr/bin/env bash
set -euo pipefail
. "$(dirname "$0")/lib.sh"

log "Linting bash scripts"
find . -type f -name "*.sh" -print0 | xargs -0 -I{} bash -n {}

if command -v shellcheck >/dev/null 2>&1; then
  log "Running shellcheck"
  find . -type f -name "*.sh" -print0 | xargs -0 shellcheck -x
else
  warn "shellcheck not found; skipping static analysis"
fi

scripts/test.sh

#!/usr/bin/env bash
set -euo pipefail
. "$(dirname "$0")/lib.sh"

log "Running unit tests"

add() { echo $(( $1 + $2 )); }
assert_eq(){ [ "$1" = "$2" ] || fail "assert_eq failed: got=$1 want=$2"; }

assert_eq "$(add 2 3)" 5
log "Tests passed"

3) Build an artifact

Produce a Docker image when Docker is available; otherwise create a tarball so downstream steps still have a stable artifact.

scripts/build.sh

#!/usr/bin/env bash
set -euo pipefail
. "$(dirname "$0")/lib.sh"

APP_NAME=${APP_NAME:-demoapp}
IMAGE_REPO=${IMAGE_REPO:-example/demoapp}
GIT_SHA=$(git rev-parse --short HEAD 2>/dev/null || echo local)
TAG=${TAG:-$GIT_SHA}
ART_DIR=${ART_DIR:-dist}

mkdir -p "$ART_DIR"

if command -v docker >/dev/null 2>&1; then
  log "Building Docker image $IMAGE_REPO:$TAG"
  docker build -t "$IMAGE_REPO:$TAG" -f docker/Dockerfile .
  echo "$IMAGE_REPO:$TAG" > "$ART_DIR/image.tag"
else
  warn "docker not found; creating tarball artifact instead"
  tar -czf "$ART_DIR/$APP_NAME-$TAG.tgz" app/
  echo "$ART_DIR/$APP_NAME-$TAG.tgz" > "$ART_DIR/artifact.path"
fi

4) Validate before you deploy

Validate manifests and compose configs in CI to fail early.

scripts/validate.sh

#!/usr/bin/env bash
set -euo pipefail
. "$(dirname "$0")/lib.sh"

STAGE=${STAGE:-staging}
COMPOSE_FILE=${COMPOSE_FILE:-docker/compose.yaml}

log "Validating deployment manifests for $STAGE"

if command -v kubectl >/dev/null 2>&1 && [ -d k8s ]; then
  require kubectl
  kubectl apply --dry-run=server -f k8s/
  kubectl diff -f k8s/ || true
elif command -v docker >/dev/null 2>&1 && [ -f "$COMPOSE_FILE" ]; then
  require docker
  docker compose -f "$COMPOSE_FILE" config >/dev/null
else
  warn "No k8s or compose config found to validate"
fi

5) Deploy safely and verify health

Deploy with a lock to prevent overlapping runs, then verify a health endpoint before marking success.

scripts/deploy.sh

#!/usr/bin/env bash
set -euo pipefail
. "$(dirname "$0")/lib.sh"

STAGE=${STAGE:-staging}
HEALTH_URL=${HEALTH_URL:-http://localhost:8080/healthz}
LOCKFILE=${LOCKFILE:-/tmp/deploy.lock}

_do_deploy() {
  log "Starting deploy to $STAGE"

  if command -v kubectl >/dev/null 2>&1 && [ -d k8s ]; then
    kubectl apply -f k8s/
    if kubectl get deploy >/dev/null 2>&1; then
      for d in $(kubectl get deploy -o name); do
        log "Waiting for $d to roll out"
        with_retry 30 2 kubectl rollout status "$d"
      done
    fi
  elif command -v docker >/dev/null 2>&1 && [ -f docker/compose.yaml ]; then
    docker compose -f docker/compose.yaml up -d --build
  else
    warn "No deploy target detected; skipping apply"
  fi

  "$(dirname "$0")/healthcheck.sh" "$HEALTH_URL" 60 2
  log "Deployment verified"
}

lock_run "$LOCKFILE" _do_deploy

scripts/healthcheck.sh

#!/usr/bin/env bash
set -euo pipefail
. "$(dirname "$0")/lib.sh"

URL=${1:-http://localhost:8080/healthz}
ATTEMPTS=${2:-30}
SLEEP=${3:-2}

require curl
with_retry "$ATTEMPTS" "$SLEEP" bash -c "curl -fsS --max-time 2 '$URL' >/dev/null"

6) Promote and rollback

Record the deployed version so rollbacks are predictable.

scripts/rollback.sh

#!/usr/bin/env bash
set -euo pipefail
. "$(dirname "$0")/lib.sh"

STAGE=${STAGE:-staging}
PREV_FILE=${PREV_FILE:-.previous_version}
CURR_FILE=${CURR_FILE:-dist/image.tag}
COMPOSE_FILE=${COMPOSE_FILE:-docker/compose.yaml}

promote() {
  [ -f "$CURR_FILE" ] || fail "Missing current version file: $CURR_FILE"
  cp "$CURR_FILE" "$PREV_FILE"
  log "Promoted version $(cat "$CURR_FILE") for $STAGE"
}

rollback() {
  [ -f "$PREV_FILE" ] || fail "No previous version recorded"
  local prev repo tag
  prev=$(cat "$PREV_FILE")
  repo=${prev%:*}
  tag=${prev#*:}

  if command -v kubectl >/dev/null 2>&1 && kubectl get deploy >/dev/null 2>&1; then
    for d in $(kubectl get deploy -o name); do
      log "Rolling back $d to image $prev"
      kubectl set image "$d" "*=$prev" --record
      with_retry 30 2 kubectl rollout status "$d"
    done
  elif command -v docker >/dev/null 2>&1 && [ -f "$COMPOSE_FILE" ]; then
    # Requires compose.yaml to use image: "${IMAGE_REPO}:${TAG}" or "${IMAGE}:${TAG}"
    IMAGE_REPO="$repo" TAG="$tag" docker compose -f "$COMPOSE_FILE" up -d
    log "Compose services updated to $repo:$tag"
  else
    fail "No supported platform for rollback"
  fi
}

case "${1:-}" in
  promote) promote ;;
  rollback) rollback ;;
  *) echo "Usage: $0 {promote|rollback}"; exit 2 ;;
esac

Tip: Kubernetes also supports kubectl rollout undo deploy/<name> to revert to the previous ReplicaSet without managing image tags. Using explicit tags plus a recorded file keeps history portable across clusters and runners.

7) CI integration

GitHub Actions (/.github/workflows/ci.yaml):

name: ci
on:
  push:
    branches: [ main ]

jobs:
  build-test-deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Lint
        run: scripts/lint.sh
      - name: Test
        run: scripts/test.sh
      - name: Build
        run: scripts/build.sh
      - name: Validate manifests
        env:
          STAGE: staging
        run: scripts/validate.sh
      - name: Deploy to staging and verify
        env:
          STAGE: staging
          HEALTH_URL: https://example-staging.local/healthz
        run: scripts/deploy.sh
      - name: Record version for rollback
        run: scripts/rollback.sh promote

GitLab CI (/.gitlab-ci.yml):

stages: [lint, test, build, validate, deploy]

variables:
  GIT_STRATEGY: fetch

lint:
  stage: lint
  image: alpine:3.20
  before_script: ["apk add --no-cache bash findutils shellcheck"]
  script:
    - chmod +x scripts/*.sh
    - scripts/lint.sh

unit_test:
  stage: test
  image: alpine:3.20
  before_script: ["apk add --no-cache bash"]
  script:
    - chmod +x scripts/*.sh
    - scripts/test.sh

build:
  stage: build
  image: docker:26
  services:
    - docker:26-dind
  variables:
    DOCKER_HOST: tcp://docker:2375
    DOCKER_TLS_CERTDIR: ""
  script:
    - chmod +x scripts/*.sh
    - scripts/build.sh
  artifacts:
    paths:
      - dist/

validate:
  stage: validate
  image: bitnami/kubectl:1.29
  script:
    - chmod +x scripts/*.sh
    - STAGE=staging scripts/validate.sh

deploy_staging:
  stage: deploy
  image: bitnami/kubectl:1.29
  when: manual
  script:
    - chmod +x scripts/*.sh
    - STAGE=staging HEALTH_URL=https://example-staging.local/healthz scripts/deploy.sh
    - scripts/rollback.sh promote

8) Comparison tables

Pipeline stages and goals:

Rollback approaches:

9) Troubleshooting and hardening tips

• Script stops without context: export TRACE=1 to enable set -x tracing from lib.sh.
• Hidden failures in pipelines: always set -euo pipefail and quote variables ("${var}").
• Health check flapping: increase with_retry attempts or SLEEP; ensure your app returns 200 OK.
• Kubectl not configured: provide KUBECONFIG or run kubectl config set-context before deploy.
• Docker permission denied: ensure runner has access to Docker socket or use DinD with DOCKER_HOST.
• Secrets in logs: never echo secret env vars; prefer files or masked CI variables.
• Concurrent deploys: ensure lock_run uses a shared path (e.g., /tmp/deploy.lock) across steps.
• Rollback fails on Compose: confirm compose.yaml uses image: "${IMAGE_REPO}:${TAG}" and that IMAGE_REPO/TAG are set from the recorded file.
• Long-running commands: wrap with timebox 60 <cmd>; install coreutils/timeout if missing.

10) Local pilot you can run today

• Make scripts executable: chmod +x scripts/*.sh
• Optional .env for local runs:
• APP_NAME=demoapp
• IMAGE_REPO=example/demoapp
• STAGE=staging
• HEALTH_URL=http://localhost:8080/healthz
• Run the flow:
• scripts/lint.sh
• scripts/test.sh
• scripts/build.sh
• scripts/validate.sh || true
• scripts/deploy.sh
• scripts/rollback.sh promote
• Test rollback:
• Deploy a bad tag (e.g., edit k8s/deployment.yaml or set TAG=bad and run deploy)
• Observe failed health check
• Run scripts/rollback.sh rollback and confirm recovery

Track: total runtime, health-check success rate during deploy, and mean time to rollback.

Conclusion

You now have a practical Bash toolkit for CI/CD: strict shell safety, reusable logging and helpers, validation before deploy, verified rollouts, and fast, auditable rollbacks. Start with the local pilot, wire it into GitHub Actions or GitLab CI, and evolve the same scripts across staging and production. Rehearse failures regularly, keep previous versions recorded in durable storage, and your Bash automation will remain predictable, inspectable, and safe to extend as your platform grows.