## Intro

Modern software teams rely on containers to package and ship applications, but the convenience of containers introduces security considerations that are easy to overlook. Docker images often bundle operating system libraries, language runtimes, and application dependencies; a single unfixed vulnerability or overly permissive configuration can expose an entire service. Integrating security checks directly into CI/CD automation catches these issues before they reach production and turns ad hoc security reviews into a repeatable, fast feedback loop.

This guide walks through a practical Docker Security CI/CD automation setup using free, widely available tooling. You will see how to inventory the environment, apply safe configuration choices, verify that checks work as expected, handle failures without breaking the pipeline, and operate the process day to day. The examples are intentionally narrow and local-first, so you can adapt them to your existing tooling without a large up-front investment.

## Version and Environment Inventory

Before adding security automation, document the versions and topology you will work with. This inventory prevents version mismatches between local development and CI runners and makes troubleshooting easier. For this guide, the reference environment is:

- Docker Engine 24.0.5 or newer, with Buildx available (run `docker version` and `docker buildx version` to confirm).
- A CI platform that supports containerised jobs, such as GitHub Actions, GitLab CI, or a self-hosted runner with Docker installed. The examples use GitHub Actions syntax, but the concepts transfer to any runner that can execute docker commands.
- One application repository containing a Dockerfile, application code, and a CI workflow definition.
- Common command-line security scanners installed in the CI job: Trivy (version 0.45 or later), Hadolint (2.12.0 or later), and Dockle (0.4.14 or later). These are single binaries and easy to pin in the pipeline.

A typical topology is a single stage build-and-scan job that builds the image, runs security scans, and only then pushes the image to a registry. Keeping scanning in one stage reduces moving parts. If your organisation requires separation of duties, you may have a later promote stage, but for the first pilot, one stage is enough.

Prerequisites for the CI runner:

- Docker daemon available to the job. On GitHub Actions, using `ubuntu-latest` runners gives you Docker out of the box.
- Network access to pull scanner images or download scanner binaries. If your CI environment is offline, pre-install scanners on a custom runner image.
- Credentials stored as secrets for the container registry. Never hard-code registry credentials in the workflow file.

Check these prerequisites with a minimal job:

```yaml
name: environment-check
on: [push]
jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - name: Check Docker
        run: docker version --format '{{.Server.Version}}'
      - name: Check Buildx
        run: docker buildx version
```

Expected output for the first step is a single server version line, such as `24.0.5`. The second step prints the Buildx version, for example `github.com/docker/buildx v0.11.2`. If either command fails, fix the runner before proceeding.

## Safe Configuration Path

Start with a narrow, measurable security check that is easy to inspect locally. Attempting to enable every scanner and policy at once often produces false positives and slows down the pipeline. Begin with three checks that cover different layers:

- Dockerfile linting with Hadolint to catch common misconfigurations like using `latest` tags or running as root.
- Image vulnerability scanning with Trivy to identify known CVEs in the built image.
- CIS benchmark checks with Dockle to validate image compliance against best practices.

These tools run locally on a developer machine, so you can reproduce CI results exactly. Configure each scanner with a clear policy file instead of relying on defaults, and keep the policy in the repository so changes are reviewed.

### Repository Layout

Add a `security/` directory to the application repository:

```
security/
  hadolint.yaml
  trivy.yaml
  dockle.config
```

An example Hadolint configuration that ignores a common style rule and fails on a specific instruction:

```yaml
# security/hadolint.yaml
ignored:
  - DL3008 # Pin versions in apt install; we use a base image with pinned packages
failure-threshold: warning
```

Trivy's configuration can set a severity threshold so only critical and high vulnerabilities fail the build:

```yaml
# security/trivy.yaml
severity:
  - CRITICAL
  - HIGH
```

Dockle uses a `--config` flag with a simple JSON file to disable checks that are not relevant:

```json
{
  "ignore": [
    "CIS-DI-0001"
  ]
}
```

### CI Workflow Definition

A GitHub Actions workflow that builds the image locally for scanning with `docker/build-push-action`, runs all three scanners, and only pushes if all pass using `docker/build-push-action` again with `push: true`. The workflow is designed to fail fast: if any scan fails, the push step is skipped.

```yaml
name: docker-security
on:
  push:
    branches: [main]
  pull_request:

jobs:
  build-and-scan:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Lint Dockerfile
        run: |
          docker run --rm -i hadolint/hadolint:2.12.0-alpine \
            hadolint --config /security/hadolint.yaml - < Dockerfile

      - name: Build image for scanning
        uses: docker/build-push-action@v5
        with:
          context: .
          load: true
          push: false
          tags: myapp:${{ github.sha }}

      - name: Scan image with Trivy
        run: |
          docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
            aquasec/trivy:0.45.0 image --config /security/trivy.yaml \
            myapp:${{ github.sha }}

      - name: Check CIS benchmarks with Dockle
        run: |
          docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
            goodwithtech/dockle:0.4.14 --config /security/dockle.config \
            myapp:${{ github.sha }}

      - name: Login to registry
        if: success()
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.REGISTRY_USERNAME }}
          password: ${{ secrets.REGISTRY_PASSWORD }}

      - name: Push image
        if: success()
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: myapp:${{ github.sha }}
```

Note how the workflow mounts the Docker socket for Trivy and Dockle. This is necessary for them to inspect the local image. In a hardened environment, consider using a scanner that pulls the image from a registry after push to avoid socket access, but for a first pilot the socket mount is acceptable and keeps the feedback loop fast.

### Local Reproduction

Every CI step can run locally to validate changes before pushing. For example, the Hadolint step:

```bash
docker run --rm -i hadolint/hadolint:2.12.0-alpine \
  hadolint --config security/hadolint.yaml - < Dockerfile
```

If the Dockerfile has an issue like `FROM ubuntu: latest`, Hadolint prints a warning or error depending on severity. Fix the issue and re-run until clean.

## Verification and Diagnostics

After the pipeline is in place, verify that each check behaves as intended. Start with a deliberately vulnerable image to confirm scanners detect problems, then switch to the real application and confirm a clean result.

### Test with a Known-Bad Image

Create a temporary `Dockerfile.bad` in a scratch directory:

```dockerfile
FROM ubuntu: latest
RUN apt-get update && apt-get install -y curl
USER root
```

Run each scanner manually:

```bash
# Hadolint
docker run --rm -i hadolint/hadolint:2.12.0-alpine hadolint - < Dockerfile.bad
# Expected: multiple warnings, e.g. DL3007 (latest tag), DL3015 (apt install without -y --no-install-recommends), DL3002 (USER root)

# Build the bad image for Trivy and Dockle
docker build -t bad-image -f Dockerfile.bad .

# Trivy (with default high severity threshold)
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock aquasec/trivy:0.45.0 image bad-image
# Expected: list of vulnerabilities with severity HIGH or CRITICAL, exit code 1

# Dockle
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock goodwithtech/dockle:0.4.14 bad-image
# Expected: FAIL messages for checks like CIS-DI-0005 (avoid latest tag), CIS-DI-0006 (add HEALTHCHECK), etc., exit code 1
```

### Verify Real Application Clean Result

Once the real application Dockerfile is fixed to pass the configured policies, run the same scanners and expect exit code 0. For example, a well-formed Dockerfile might be:

```dockerfile
FROM node:20-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
USER node
EXPOSE 3000
CMD ["node", "server.js"]
```

Run the Trivy scan and observe the output:

```
myapp: latest (alpine 3.18.4)
Total: 0 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 0, CRITICAL: 0)
```

A zero-vulnerability result means the scanner ran successfully, not that the image is perfectly secure, but it confirms the tooling is wired correctly.

### Pipeline Logs and Exit Codes

In your CI logs, each scan should complete with a clear pass/fail. GitHub Actions shows a red X on failure and annotations such as `Process completed with exit code 1.`. If a scan is skipped due to a previous failure, the push step will be skipped, and the pipeline stops. This is the desired fail-fast behaviour.

## Failure Modes and Recovery

Security automation can fail for reasons unrelated to actual vulnerabilities. Knowing these failure modes ahead of time reduces downtime and frustration.

### Common Failure Modes

| Failure | Typical Cause | Immediate Action |
|---|---|---|
| Scanner container fails to start | Network issue pulling scanner image or invalid scanner version | Re-run job; if persistent, pin a known-good scanner image and pre-pull on runner |
| Docker daemon not available | Runner does not have Docker running | Use a runner image with Docker, or start Docker service in the job |
| Scan finds vulnerabilities | New CVE published or dependency added | Update base image or dependency, re-run scan |
| Policy misconfiguration | Scanner ignores a check or requires missing config file | Validate config file syntax locally, correct path in CI |
| False positive | Scanner flags a package not exploitable in context | Add ignore rule with justification in policy file |

### Rollback and Recovery Steps

If a security check change breaks the pipeline and blocks merges, you can temporarily disable the failing check while you fix the underlying issue. For example, to make Trivy only fail on critical vulnerabilities during an incident, modify `security/trivy.yaml`:

```yaml
severity:
  - CRITICAL
```

Commit this change, and the pipeline will pass for high-severity findings. Revert the change after the incident is resolved.

If a scanner version update causes unexpected failures, pin the scanner to the last known working version. In the workflow, change `aquasec/trivy:0.45.0` to `aquasec/trivy:0.44.1` and retry.

For complete pipeline failure due to infrastructure issues, you can always build and scan locally with the same commands and push manually if urgent, but this should be a rare exception.

### Recovery Verification

After any change to scanner policy or tooling, verify with a known-bad image that the scanner still detects relevant issues, then verify the real application passes. This two-step check prevents a scanner from being silently disabled.

## Common Pitfalls and How to Avoid Them

Many teams hit the same obstacles when first automating Docker security checks. Being aware of them saves time and frustration.

### Pitfall 1: Running scanners without pinned versions

**Why it happens:** Teams often use the `latest` tag for scanner images because it is convenient. But scanner behavior and flags can change between versions, causing unpredictable pipeline failures.

**How to avoid:** Always pin the scanner image to a specific version digest, not just a tag. For example, use `aquasec/trivy:0.45.0@sha256:...` to ensure immutable behavior. Re-evaluate the digest monthly or when a new scanner release is needed.

### Pitfall 2: Overly permissive policy files

**Why it happens:** To reduce false positives, teams sometimes disable too many checks or lower severity thresholds too much, which can hide real issues.

**How to avoid:** Start with the recommended defaults from the scanner documentation. Only add ignore rules with a clear justification in the policy file comment. Review all ignore rules during the monthly audit and remove any that are no longer needed.

### Pitfall 3: Not verifying that scanners actually detect issues

**Why it happens:** Teams assume that if the pipeline passes, the scanners are working. But a misconfigured scanner can silently skip checks.

**How to avoid:** Periodically (at least monthly) run a known-bad image through the pipeline to confirm scanners fail as expected. This can be automated with a scheduled CI job that builds a deliberately vulnerable image and asserts failure.

### Pitfall 4: Docker socket mount in CI

**Why it happens:** Mounting the Docker socket into scanner containers is convenient but gives the scanner full control over the Docker daemon, which can be a security risk in shared CI environments.

**How to avoid:** Use a dedicated CI runner for security scanning, or switch to scanners that can pull images from a registry after push. If you must use the socket, restrict the scanner container's capabilities and run it with a non-root user when possible.

### Pitfall 5: Ignoring scanner warnings about base image updates

**Why it happens:** Teams often pin base images to a specific tag and forget to update them, leaving known vulnerabilities unpatched for months.

**How to avoid:** Set up a scheduled CI job (e.g., weekly) that builds the image with the latest base image tag and runs scanners. If vulnerabilities are found, create a ticket to update the base image. Alternatively, use automated dependency update tools like Renovate or Dependabot to keep base images current.

## Operations Checklist

Use this checklist for initial setup and periodic review.

### Initial Setup

- [ ] Document Docker, runner, and scanner versions in repository README or docs.
- [ ] Add security policy files (`hadolint.yaml`, `trivy.yaml`, `dockle.config`) to repository.
- [ ] Create CI workflow with build, scan, push steps.
- [ ] Test workflow with a deliberately vulnerable image to confirm failures.
- [ ] Fix application Dockerfile to pass all checks.
- [ ] Verify pipeline succeeds on a pull request and main branch.

### Weekly Review (Owner: Security Champion - for example, Alex Chen, DevOps Engineer)

- [ ] Review scanner output for new vulnerabilities; triage high and critical findings. - **Owner:** Alex Chen, DevOps Engineer - **Frequency:** Every Monday morning
- [ ] Check pipeline run history for flaky failures. - **Owner:** Alex Chen, DevOps Engineer - **Frequency:** Every Monday morning
- [ ] Update scanner versions to latest stable (test locally first). - **Owner:** Alex Chen, DevOps Engineer - **Frequency:** Every Monday morning, but only if local tests pass
- [ ] Review policy exceptions; remove any no longer needed. - **Owner:** Alex Chen, DevOps Engineer - **Frequency:** Every Monday morning

### Monthly Audit (Owner: Platform Security Lead - for example, Maria Rodriguez, Security Engineering Manager)

- [ ] Re-run entire security suite on a random historical image from registry to ensure scanners still detect known issues. - **Owner:** Maria Rodriguez, Security Engineering Manager - **Frequency:** First Monday of each month
- [ ] Rotate registry credentials. - **Owner:** Maria Rodriguez, Security Engineering Manager - **Frequency:** First Monday of each month
- [ ] Validate that the CI workflow has not been altered without review. - **Owner:** Maria Rodriguez, Security Engineering Manager - **Frequency:** First Monday of each month

## Conclusion

Automating Docker Security checks in CI/CD is a manageable first step toward a more secure supply chain. By inventorying the environment, starting with a narrow set of scanners, verifying detection with a bad image, and preparing for common failures, you can integrate security into every build without slowing development to a crawl. The examples in this guide use free tools and local reproducibility, so you can implement the process in an afternoon and extend it later with additional policies, signing, or admission control. Remember to keep the feedback loop fast and the policies clear; a security pipeline that developers trust and understand is the one that gets used.
