Docker Multi-Stage Builds in CI/CD: A Hands-On Automation Guide

Docker multi-stage builds are a cornerstone technique for creating lean, production-ready container images by cleanly separating the build environment from the runtime environment. When integrated into CI/CD pipelines, they unlock automated, repeatable, and efficient image creation. This guide provides a heavily hands-on walkthrough: setting up your environment, writing a safe multi-stage Dockerfile, configuring pipelines in GitHub Actions and GitLab CI, verifying builds, recovering from common failures, and maintaining the process with an operations checklist.

Intro

Modern software delivery demands speed, reliability, and security. Docker multi-stage builds address all three by letting developers use heavyweight build tools in intermediate stages, then copy only the necessary artifacts into a minimal final image. Automating these builds in CI/CD removes manual steps, ensures consistency across environments, and makes every build auditable.

This article is intentionally practical. You will inventory your environment, write and test a multi-stage Dockerfile, configure a CI/CD pipeline with cache optimizations, verify the resulting image, handle common failure modes, and adopt a rollout checklist. Concrete commands, configuration snippets, and expected outputs are provided at every step, so you can follow along in your own repositories.

Version and Environment Inventory

Before automating multi-stage builds, confirm that your environment meets the requirements. This section lists the necessary components, versions, and typical topology. The owner for the environment configuration should be a designated DevOps or platform engineer, reviewed quarterly.

Prerequisites

Multi-stage builds require Docker 17.05 or higher, but BuildKit provides advanced layer caching and parallel stage execution. BuildKit is enabled by default in Docker 23.0+; for older versions, set DOCKER_BUILDKIT=1 in your environment or CI runner.

Topology

A typical setup includes the following components:

• Developer workstation: For local testing of the Dockerfile and application.
• CI/CD runner: Executes pipeline jobs; must have Docker installed or access to a Docker daemon (e.g., Docker-in-Docker for GitLab).
• Registry: Stores built images for deployment.
• Deployment target: Where the final image runs (Kubernetes, VM, serverless platform, etc.).

Verify Docker and BuildKit on your runner with these commands:

docker version --format '{{.Server.Version}}'
# Example output: 24.0.7

docker buildx version
# Example output: github.com/docker/buildx v0.11.2

If docker buildx version returns an error, install the buildx plugin or use a runner image that includes it. All CI examples in this guide assume BuildKit is available.

Safe Configuration Path

Start with a minimal, inspectable setup before scaling to multiple services. The following steps guide you through creating a multi-stage Dockerfile and wiring it into CI/CD.

Step 1: Create a Multi-Stage Dockerfile

Consider a Node.js application. The first stage installs all dependencies and builds the application; the second stage copies only the production artifacts and runtime dependencies, leaving build tools behind.

Save this as Dockerfile in your repository root:

# syntax=docker/dockerfile:1
# Stage 1: Build
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# Stage 2: Production
FROM node:18-alpine
ENV NODE_ENV=production
WORKDIR /app
COPY --from=builder /app/package*.json ./
RUN npm ci --only=production
COPY --from=builder /app/dist ./dist
EXPOSE 3000
CMD ["node", "dist/server.js"]

Key points:

• The AS builder stage declares an alias used in COPY --from=builder.
• npm ci is used instead of npm install for reproducible installs based on the lockfile.
• The final stage copies package*.json and runs npm ci --only=production, then copies only the compiled dist directory. This keeps node_modules lean and excludes dev dependencies.
• node:18-alpine is used in both stages, but the production stage could switch to an even smaller runtime image like node:18-alpine (already minimal) or distroless if your app supports it.

Step 2: Test Locally

Build the image locally to ensure the Dockerfile works before touching CI:

docker build -t myapp:test .
# Expected output (truncated):
# => [builder 1/5] FROM docker.io/library/node:18-alpine
# => [builder 2/5] WORKDIR /app
# => [builder 3/5] COPY package*.json ./
# => [builder 4/5] RUN npm ci
# => [builder 5/5] COPY . .
# => [builder 6/6] RUN npm run build
# => [stage-1 1/4] FROM docker.io/library/node:18-alpine
# => [stage-1 2/4] WORKDIR /app
# => [stage-1 3/4] COPY --from=builder /app/package*.json ./
# => [stage-1 4/4] RUN npm ci --only=production
# => exporting to image
# => => naming to docker.io/library/myapp:test

The key indicator is the COPY --from=builder line appearing in the second stage. If that line fails, double-check the source path inside the builder stage.

Run a container from the image and verify the app responds:

docker run -p 3000:3000 myapp:test
# Expected: Server started on port 3000

If you see an error like Error: Cannot find module, you likely missed a runtime dependency in the final stage.

Step 3: Configure CI/CD Pipeline

Now automate the build. Below are complete examples for GitHub Actions and GitLab CI.

GitHub Actions

Create .github/workflows/build.yml:

name: Build and Push
on:
  push:
    branches: [ main ]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
      - name: Login to Registry
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: ghcr.io/${{ github.repository }}:latest
          cache-from: type=gha
          cache-to: type=gha, mode=max

This workflow triggers on pushes to main, checks out the code, sets up BuildKit, logs into GitHub Container Registry using the built-in GITHUB_TOKEN, and builds and pushes the image. The cache-from and cache-to options enable GitHub Actions cache for layer caching, which dramatically speeds up subsequent builds.

Reviewer note: For production, consider using a dedicated secret like GHCR_PAT instead of GITHUB_TOKEN if you need broader permissions, and avoid pushing latest in a strict environment; use commit SHA or version tags instead.

GitLab CI

Create .gitlab-ci.yml:

image: docker:24
services:
  - docker:24-dind
variables:
  DOCKER_BUILDKIT: 1
stages:
  - build
build-image:
  stage: build
  script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker build -t $CI_REGISTRY_IMAGE:latest .
    - docker push $CI_REGISTRY_IMAGE:latest

This uses Docker-in-Docker (dind) to provide a Docker daemon inside the runner. The DOCKER_BUILDKIT: 1 variable enables BuildKit. After staging, it logs into the GitLab Container Registry and builds/pushes. To use GitLab's cache, add a cache block with key: build-cache and paths: [.cache] or use --cache-from with a registry-stored cache image.

Owner: The pipeline configuration should be owned by the platform team lead, with changes reviewed via pull request.

Step 4: Push to Registry and Verify

After a successful pipeline run, the image is available in your registry. Pull and run it locally to confirm:

docker pull ghcr.io/yourname/yourrepo:latest
docker run -p 3000:3000 ghcr.io/yourname/yourrepo:latest
# Expected: Server started on port 3000

If the image fails to run, go back to local testing to isolate whether the issue is in the Dockerfile or the pipeline.

Verification and Diagnostics

Automated checks are essential to ensure builds are correct and efficient. This section covers post-build verification and diagnostic commands you can run in the pipeline or locally.

Image Size Comparison

Multi-stage builds should produce significantly smaller images than single-stage builds. Compare sizes:

docker images myapp
# Example output:
# REPOSITORY   TAG       IMAGE ID       CREATED          SIZE
# myapp        test      abc123def456   2 minutes ago    150MB

A single-stage Node build with all dev dependencies would typically be 400MB or more. A 150MB image suggests the multi-stage approach is working, though you may still be able to trim further (e.g., using distroless base images).

If the image is unexpectedly large, inspect the Dockerfile for missed COPY --from directives or for unnecessary files being copied (e.g., .dockerignore not excluding node_modules from the build context).

Inspect Image Layers

Use docker history to see layer sizes and confirm only necessary files are included:

docker history myapp:test
# Example output (truncated):
# IMAGE          CREATED          CREATED BY                                      SIZE
# abc123def456   2 minutes ago    CMD ["node", "dist/server.js"]                 0B
# 789ghi012jkl   2 minutes ago    EXPOSE map[3000/tcp:{}]                       0B
# 456mno345pqr   2 minutes ago    COPY /app/dist ./dist # buildkit                12MB
# 123stu456vwx   2 minutes ago    RUN /bin/sh -c npm ci --only=production # b...  80MB
# ...

Look for unusually large layers. The npm ci --only=production layer may still be large if dependencies are heavy, but it should not include dev tools like webpack or test frameworks.

Run Container and Check Logs

Start the container in detached mode and inspect logs:

docker run -d --name myapp-test -p 3000:3000 myapp:test
docker logs myapp-test
# Expected: "Server started on port 3000"

If the container exits immediately, run without -d to see the error in the foreground, or use docker logs to view the crash message.

Verify Application Endpoint

Use curl inside the container network or from the host to test the service:

curl http://localhost:3000/health
# Expected: {"status":"ok"}

For a more rigorous check, add a smoke test stage to your CI pipeline after the build. For example, in GitHub Actions, add a step that runs the container and curls the endpoint before pushing or after pulling.

Pipeline Logs

In CI/CD, review build logs for warnings. Look for:

• Dependency installation warnings (e.g., deprecated packages).
• Unused build arguments.
• Large layer sizes reported by BuildKit.
• Cache misses (if you expected cache hits).

For GitHub Actions, check the "Build and push" step logs. For GitLab, review job logs in the pipeline view. Cache hit/miss information is often displayed when using cache-from and cache-to.

Common Pitfalls and How to Avoid Them

Even experienced teams make mistakes with multi-stage builds in CI/CD. Here are the most frequent pitfalls, why they occur, and concrete recovery steps.

1. Not Using a .dockerignore File

Why it happens: Developers assume Docker only copies what is needed, but the build context includes all files in the directory (unless excluded). This can bloat the context, slow builds, and even leak secrets.

How to avoid: Create a .dockerignore in the repository root before building. A typical Node.js .dockerignore:

node_modules
npm-debug.log
.git
.env
*.md

This prevents local node_modules from overwriting the container install and keeps secrets out of the image.

2. Missing Lockfile in Build Stage

Why it happens: Developers use npm install instead of npm ci, or forget to commit package-lock.json.

How to avoid: Always commit the lockfile. In the Dockerfile, use npm ci in both stages. If you forget, the build may succeed but with non-reproducible dependencies, or fail with a clear error about missing lockfile.

Recovery: If the lockfile is missing, generate it locally (npm install --package-lock-only), commit it, and rerun the pipeline.

3. Copying from the Wrong Stage or Wrong Path

Why it happens: Typos in stage aliases or incorrect paths inside the builder stage. For example, COPY --from=builder /app/dist ./dist fails if the build output is in /app/build.

How to avoid: Test locally first. Use docker build --progress=plain to see exact copy commands. In CI, the same flag can be set with build-args or environment variables if using raw docker build.

Recovery: Examine the failing layer log, correct the path, and commit the fix.

4. Ignoring Build Cache Configuration

Why it happens: Teams rely on default caching, which may not persist between CI runs, leading to slow builds and unnecessary bandwidth.

How to avoid: Explicitly configure caching. In GitHub Actions, use cache-from: type=gha and cache-to: type=gha, mode=max. In GitLab, use --cache-from with a registry-stored cache image (e.g., docker build --cache-from $CI_REGISTRY_IMAGE:cache).

Recovery: Verify cache hits by looking for "CACHED" in the build log. If no hits, ensure the cache is actually pushed and pulled and that the Dockerfile ordering does not invalidate cache early (e.g., COPY . . before RUN npm ci).

5. Running as Root in the Final Stage

Why it happens: Using the default root user is common but violates least privilege and can be a security risk.

How to avoid: Add a non-root user in the final stage:

FROM node:18-alpine
RUN addgroup -S appgroup && adduser -S appuser -G appgroup
USER appuser
WORKDIR /app
COPY --from=builder /app/package*.json ./
RUN npm ci --only=production
COPY --from=builder /app/dist ./dist
EXPOSE 3000
CMD ["node", "dist/server.js"]

This reduces the attack surface if the container is compromised.

6. Not Tagging Images Properly

Why it happens: Using only latest makes rollback and auditing difficult.

How to avoid: Tag images with a unique identifier such as the commit SHA or a semantic version. In GitHub Actions, you can set tags: ghcr.io/${{ github.repository }}:${{ github.sha }} along with latest. In GitLab, use $CI_COMMIT_SHORT_SHA.

Recovery: If you only have latest, you may be able to re-tag an older image from the registry if you know its digest, but it is far better to implement proper tagging now.

Failure Modes and Recovery

Even with careful setup, failures occur. This section identifies common issues and provides concrete recovery steps.

Common Failure Modes

Rollback Strategies

If a new image is faulty, roll back to a previous known-good image using one of these strategies:

Tag-based rollback: Keep version tags in the registry. Deploy using a specific tag, not latest.

docker pull ghcr.io/yourname/yourrepo:v1.2.3
docker run -p 3000:3000 ghcr.io/yourname/yourrepo:v1.2.3

Registry rollback: Most registries support retention policies and image digests. Use a previous tag or re-tag a known-good image using its digest:

docker tag ghcr.io/yourname/yourrepo@sha256:abc123... ghcr.io/yourname/yourrepo:rollback

Pipeline rollback: Revert the commit that introduced the problematic Dockerfile or pipeline change, then rerun the pipeline.

Owner: The release manager or on-call engineer owns the rollback decision, executed within 15 minutes of detecting the failure.

Recovery Checks

After rollback, verify the following:

• Container runs with expected version (check docker inspect or app version endpoint).
• Logs show no errors.
• Health endpoint responds correctly.
• CI/CD pipeline passes on a test commit.
• Monitoring dashboards show normal metrics.

Operations Checklist

Use this checklist to ensure continued smooth operation of multi-stage builds in CI/CD. The platform engineer should review this checklist monthly and update it as needed.

• [ ] Dockerfile uses multi-stage syntax and is versioned in source control.
• [ ] Local build succeeds with docker build.
• [ ] CI/CD pipeline is configured and triggers on appropriate events (e.g., push to main, pull requests).
• [ ] Registry credentials are stored securely (e.g., GitHub Secrets, GitLab CI variables).
• [ ] Build cache is enabled and effective; check pipeline logs for cache hits.
• [ ] Image size is within expected range (e.g., under 200MB for a typical Node service); size reported in pipeline artifacts or logs.
• [ ] Container startup is tested in pipeline via a smoke test step.
• [ ] Rollback procedure is documented and tested at least once.
• [ ] Regular review of Dockerfile for updates and security patches (e.g., base image updates monthly).
• [ ] Monitoring and alerting for registry and deployment failures.
• [ ] Non-root user configured in final stage.
• [ ] Immutable tags used for production, with latest only for development convenience.

Conclusion

Automating Docker multi-stage builds in CI/CD streamlines the path from code to deployable artifact. By following the steps in this guide, you can create lean images, reduce build times with caching, and maintain a reliable pipeline.

Start with a narrow pilot, as recommended: pick a single service, implement the Dockerfile and pipeline, verify thoroughly, and then expand to other services. Remember to continuously monitor and iterate. Assign clear ownership for the pipeline configuration and rollback procedures to avoid ambiguity.

Next steps:

• Apply the multi-stage Dockerfile to one of your applications.
• Set up the CI/CD pipeline and push to a registry.
• Run the verification commands and review the checklist.
• Document and test your rollback plan.
• Share the approach with your team and gather feedback.

By adopting these practices, you will enhance your team's delivery efficiency and reduce production risks.