Docker multi-stage builds are a cornerstone technique for creating lean, production-ready container images by cleanly separating the build environment from the runtime environment. When integrated into CI/CD pipelines, they unlock automated, repeatable, and efficient image creation. This guide provides a heavily hands-on walkthrough: setting up your environment, writing a safe multi-stage Dockerfile, configuring pipelines in GitHub Actions and GitLab CI, verifying builds, recovering from common failures, and maintaining the process with an operations checklist. ## Intro Modern software delivery demands speed, reliability, and security. Docker multi-stage builds address all three by letting developers use heavyweight build tools in intermediate stages, then copy only the necessary artifacts into a minimal final image. Automating these builds in CI/CD removes manual steps, ensures consistency across environments, and makes every build auditable. This article is intentionally practical. You will inventory your environment, write and test a multi-stage Dockerfile, configure a CI/CD pipeline with cache optimizations, verify the resulting image, handle common failure modes, and adopt a rollout checklist. Concrete commands, configuration snippets, and expected outputs are provided at every step, so you can follow along in your own repositories. ## Version and Environment Inventory Before automating multi-stage builds, confirm that your environment meets the requirements. This section lists the necessary components, versions, and typical topology. The owner for the environment configuration should be a designated DevOps or platform engineer, reviewed quarterly. ### Prerequisites | Component | Recommended Version | Purpose | |---|---|---| | Docker Engine | 20.10+ with BuildKit enabled | Building images with multi-stage support | | CI/CD Platform | GitHub Actions, GitLab CI, Jenkins, or equivalent | Automating pipeline execution | | Container Registry | Docker Hub, AWS ECR, GCR, GHCR, etc. | Storing built images | | Source Repository | Git-based (GitHub, GitLab, Bitbucket) | Version control for Dockerfile and app code | | Base Images | e.g., `node:18-alpine`, `golang:1.21-alpine` | Build stage dependencies | Multi-stage builds require Docker 17.05 or higher, but BuildKit provides advanced layer caching and parallel stage execution. BuildKit is enabled by default in Docker 23.0+; for older versions, set `DOCKER_BUILDKIT=1` in your environment or CI runner. ### Topology A typical setup includes the following components: - **Developer workstation**: For local testing of the Dockerfile and application. - **CI/CD runner**: Executes pipeline jobs; must have Docker installed or access to a Docker daemon (e.g., Docker-in-Docker for GitLab). - **Registry**: Stores built images for deployment. - **Deployment target**: Where the final image runs (Kubernetes, VM, serverless platform, etc.). Verify Docker and BuildKit on your runner with these commands: ```bash docker version --format '{{.Server.Version}}' # Example output: 24.0.7 docker buildx version # Example output: github.com/docker/buildx v0.11.2 ``` If `docker buildx version` returns an error, install the buildx plugin or use a runner image that includes it. All CI examples in this guide assume BuildKit is available. ## Safe Configuration Path Start with a minimal, inspectable setup before scaling to multiple services. The following steps guide you through creating a multi-stage Dockerfile and wiring it into CI/CD. ### Step 1: Create a Multi-Stage Dockerfile Consider a Node.js application. The first stage installs all dependencies and builds the application; the second stage copies only the production artifacts and runtime dependencies, leaving build tools behind. Save this as `Dockerfile` in your repository root: ```dockerfile # syntax=docker/dockerfile:1 # Stage 1: Build FROM node:18-alpine AS builder WORKDIR /app COPY package*.json ./ RUN npm ci COPY . . RUN npm run build # Stage 2: Production FROM node:18-alpine ENV NODE_ENV=production WORKDIR /app COPY --from=builder /app/package*.json ./ RUN npm ci --only=production COPY --from=builder /app/dist ./dist EXPOSE 3000 CMD ["node", "dist/server.js"] ``` Key points: - The `AS builder` stage declares an alias used in `COPY --from=builder`. - `npm ci` is used instead of `npm install` for reproducible installs based on the lockfile. - The final stage copies `package*.json` and runs `npm ci --only=production`, then copies only the compiled `dist` directory. This keeps `node_modules` lean and excludes dev dependencies. - `node:18-alpine` is used in both stages, but the production stage could switch to an even smaller runtime image like `node:18-alpine` (already minimal) or `distroless` if your app supports it. ### Step 2: Test Locally Build the image locally to ensure the Dockerfile works before touching CI: ```bash docker build -t myapp:test . # Expected output (truncated): # => [builder 1/5] FROM docker.io/library/node:18-alpine # => [builder 2/5] WORKDIR /app # => [builder 3/5] COPY package*.json ./ # => [builder 4/5] RUN npm ci # => [builder 5/5] COPY . . # => [builder 6/6] RUN npm run build # => [stage-1 1/4] FROM docker.io/library/node:18-alpine # => [stage-1 2/4] WORKDIR /app # => [stage-1 3/4] COPY --from=builder /app/package*.json ./ # => [stage-1 4/4] RUN npm ci --only=production # => exporting to image # => => naming to docker.io/library/myapp:test ``` The key indicator is the `COPY --from=builder` line appearing in the second stage. If that line fails, double-check the source path inside the builder stage. Run a container from the image and verify the app responds: ```bash docker run -p 3000:3000 myapp:test # Expected: Server started on port 3000 ``` If you see an error like `Error: Cannot find module`, you likely missed a runtime dependency in the final stage. ### Step 3: Configure CI/CD Pipeline Now automate the build. Below are complete examples for GitHub Actions and GitLab CI. #### GitHub Actions Create `.github/workflows/build.yml`: ```yaml name: Build and Push on: push: branches: [ main ] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Set up Docker Buildx uses: docker/setup-buildx-action@v3 - name: Login to Registry uses: docker/login-action@v3 with: registry: ghcr.io username: ${{ github.actor }} password: ${{ secrets.GITHUB_TOKEN }} - name: Build and push uses: docker/build-push-action@v5 with: context: . push: true tags: ghcr.io/${{ github.repository }}:latest cache-from: type=gha cache-to: type=gha, mode=max ``` This workflow triggers on pushes to `main`, checks out the code, sets up BuildKit, logs into GitHub Container Registry using the built-in `GITHUB_TOKEN`, and builds and pushes the image. The `cache-from` and `cache-to` options enable GitHub Actions cache for layer caching, which dramatically speeds up subsequent builds. Reviewer note: For production, consider using a dedicated secret like `GHCR_PAT` instead of `GITHUB_TOKEN` if you need broader permissions, and avoid pushing `latest` in a strict environment; use commit SHA or version tags instead. #### GitLab CI Create `.gitlab-ci.yml`: ```yaml image: docker:24 services: - docker:24-dind variables: DOCKER_BUILDKIT: 1 stages: - build build-image: stage: build script: - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - docker build -t $CI_REGISTRY_IMAGE:latest . - docker push $CI_REGISTRY_IMAGE:latest ``` This uses Docker-in-Docker (`dind`) to provide a Docker daemon inside the runner. The `DOCKER_BUILDKIT: 1` variable enables BuildKit. After staging, it logs into the GitLab Container Registry and builds/pushes. To use GitLab's cache, add a `cache` block with `key: build-cache` and `paths: [.cache]` or use `--cache-from` with a registry-stored cache image. Owner: The pipeline configuration should be owned by the platform team lead, with changes reviewed via pull request. ### Step 4: Push to Registry and Verify After a successful pipeline run, the image is available in your registry. Pull and run it locally to confirm: ```bash docker pull ghcr.io/yourname/yourrepo:latest docker run -p 3000:3000 ghcr.io/yourname/yourrepo:latest # Expected: Server started on port 3000 ``` If the image fails to run, go back to local testing to isolate whether the issue is in the Dockerfile or the pipeline. ## Verification and Diagnostics Automated checks are essential to ensure builds are correct and efficient. This section covers post-build verification and diagnostic commands you can run in the pipeline or locally. ### Image Size Comparison Multi-stage builds should produce significantly smaller images than single-stage builds. Compare sizes: ```bash docker images myapp # Example output: # REPOSITORY TAG IMAGE ID CREATED SIZE # myapp test abc123def456 2 minutes ago 150MB ``` A single-stage Node build with all dev dependencies would typically be 400MB or more. A 150MB image suggests the multi-stage approach is working, though you may still be able to trim further (e.g., using `distroless` base images). If the image is unexpectedly large, inspect the Dockerfile for missed `COPY --from` directives or for unnecessary files being copied (e.g., `.dockerignore` not excluding `node_modules` from the build context). ### Inspect Image Layers Use `docker history` to see layer sizes and confirm only necessary files are included: ```bash docker history myapp:test # Example output (truncated): # IMAGE CREATED CREATED BY SIZE # abc123def456 2 minutes ago CMD ["node", "dist/server.js"] 0B # 789ghi012jkl 2 minutes ago EXPOSE map[3000/tcp:{}] 0B # 456mno345pqr 2 minutes ago COPY /app/dist ./dist # buildkit 12MB # 123stu456vwx 2 minutes ago RUN /bin/sh -c npm ci --only=production # b... 80MB # ... ``` Look for unusually large layers. The `npm ci --only=production` layer may still be large if dependencies are heavy, but it should not include dev tools like `webpack` or test frameworks. ### Run Container and Check Logs Start the container in detached mode and inspect logs: ```bash docker run -d --name myapp-test -p 3000:3000 myapp:test docker logs myapp-test # Expected: "Server started on port 3000" ``` If the container exits immediately, run without `-d` to see the error in the foreground, or use `docker logs` to view the crash message. ### Verify Application Endpoint Use `curl` inside the container network or from the host to test the service: ```bash curl http://localhost:3000/health # Expected: {"status":"ok"} ``` For a more rigorous check, add a smoke test stage to your CI pipeline after the build. For example, in GitHub Actions, add a step that runs the container and curls the endpoint before pushing or after pulling. ### Pipeline Logs In CI/CD, review build logs for warnings. Look for: - Dependency installation warnings (e.g., deprecated packages). - Unused build arguments. - Large layer sizes reported by BuildKit. - Cache misses (if you expected cache hits). For GitHub Actions, check the "Build and push" step logs. For GitLab, review job logs in the pipeline view. Cache hit/miss information is often displayed when using `cache-from` and `cache-to`. ## Common Pitfalls and How to Avoid Them Even experienced teams make mistakes with multi-stage builds in CI/CD. Here are the most frequent pitfalls, why they occur, and concrete recovery steps. ### 1. Not Using a `.dockerignore` File **Why it happens:** Developers assume Docker only copies what is needed, but the build context includes all files in the directory (unless excluded). This can bloat the context, slow builds, and even leak secrets. **How to avoid:** Create a `.dockerignore` in the repository root before building. A typical Node.js `.dockerignore`: ``` node_modules npm-debug.log .git .env *.md ``` This prevents local `node_modules` from overwriting the container install and keeps secrets out of the image. ### 2. Missing Lockfile in Build Stage **Why it happens:** Developers use `npm install` instead of `npm ci`, or forget to commit `package-lock.json`. **How to avoid:** Always commit the lockfile. In the Dockerfile, use `npm ci` in both stages. If you forget, the build may succeed but with non-reproducible dependencies, or fail with a clear error about missing lockfile. Recovery: If the lockfile is missing, generate it locally (`npm install --package-lock-only`), commit it, and rerun the pipeline. ### 3. Copying from the Wrong Stage or Wrong Path **Why it happens:** Typos in stage aliases or incorrect paths inside the builder stage. For example, `COPY --from=builder /app/dist ./dist` fails if the build output is in `/app/build`. **How to avoid:** Test locally first. Use `docker build --progress=plain` to see exact copy commands. In CI, the same flag can be set with `build-args` or environment variables if using raw `docker build`. Recovery: Examine the failing layer log, correct the path, and commit the fix. ### 4. Ignoring Build Cache Configuration **Why it happens:** Teams rely on default caching, which may not persist between CI runs, leading to slow builds and unnecessary bandwidth. **How to avoid:** Explicitly configure caching. In GitHub Actions, use `cache-from: type=gha` and `cache-to: type=gha, mode=max`. In GitLab, use `--cache-from` with a registry-stored cache image (e.g., `docker build --cache-from $CI_REGISTRY_IMAGE:cache`). Recovery: Verify cache hits by looking for "CACHED" in the build log. If no hits, ensure the cache is actually pushed and pulled and that the Dockerfile ordering does not invalidate cache early (e.g., `COPY . .` before `RUN npm ci`). ### 5. Running as Root in the Final Stage **Why it happens:** Using the default `root` user is common but violates least privilege and can be a security risk. **How to avoid:** Add a non-root user in the final stage: ```dockerfile FROM node:18-alpine RUN addgroup -S appgroup && adduser -S appuser -G appgroup USER appuser WORKDIR /app COPY --from=builder /app/package*.json ./ RUN npm ci --only=production COPY --from=builder /app/dist ./dist EXPOSE 3000 CMD ["node", "dist/server.js"] ``` This reduces the attack surface if the container is compromised. ### 6. Not Tagging Images Properly **Why it happens:** Using only `latest` makes rollback and auditing difficult. **How to avoid:** Tag images with a unique identifier such as the commit SHA or a semantic version. In GitHub Actions, you can set `tags: ghcr.io/${{ github.repository }}:${{ github.sha }}` along with `latest`. In GitLab, use `$CI_COMMIT_SHORT_SHA`. Recovery: If you only have `latest`, you may be able to re-tag an older image from the registry if you know its digest, but it is far better to implement proper tagging now. ## Failure Modes and Recovery Even with careful setup, failures occur. This section identifies common issues and provides concrete recovery steps. ### Common Failure Modes | Failure | Possible Cause | Recovery | |---|---|---| | Build fails during dependency installation | Network issue or missing lockfile | Check network connectivity, ensure lockfile is committed. Rerun job after fixing network. | | Final image missing files | Incorrect `COPY --from` path | Verify paths in Dockerfile, rebuild with `--progress=plain` to see copy commands. | | Image runs but app crashes | Missing runtime dependency or wrong entrypoint | Inspect container logs, test locally, adjust final stage. | | CI/CD cannot push to registry | Authentication failure | Reconfigure registry credentials, check secrets. For GitHub Actions, ensure `GITHUB_TOKEN` has `packages: write` permission. | | Build cache not working | BuildKit not enabled or cache configuration wrong | Ensure BuildKit is enabled, adjust cache settings. Check CI logs for cache export/import lines. | | Build context too large | No `.dockerignore` or unnecessary files | Add `.dockerignore`, optimize context. | ### Rollback Strategies If a new image is faulty, roll back to a previous known-good image using one of these strategies: **Tag-based rollback:** Keep version tags in the registry. Deploy using a specific tag, not `latest`. ```bash docker pull ghcr.io/yourname/yourrepo:v1.2.3 docker run -p 3000:3000 ghcr.io/yourname/yourrepo:v1.2.3 ``` **Registry rollback:** Most registries support retention policies and image digests. Use a previous tag or re-tag a known-good image using its digest: ```bash docker tag ghcr.io/yourname/yourrepo@sha256:abc123... ghcr.io/yourname/yourrepo:rollback ``` **Pipeline rollback:** Revert the commit that introduced the problematic Dockerfile or pipeline change, then rerun the pipeline. Owner: The release manager or on-call engineer owns the rollback decision, executed within 15 minutes of detecting the failure. ### Recovery Checks After rollback, verify the following: - Container runs with expected version (check `docker inspect` or app version endpoint). - Logs show no errors. - Health endpoint responds correctly. - CI/CD pipeline passes on a test commit. - Monitoring dashboards show normal metrics. ## Operations Checklist Use this checklist to ensure continued smooth operation of multi-stage builds in CI/CD. The platform engineer should review this checklist monthly and update it as needed. - [ ] Dockerfile uses multi-stage syntax and is versioned in source control. - [ ] Local build succeeds with `docker build`. - [ ] CI/CD pipeline is configured and triggers on appropriate events (e.g., push to main, pull requests). - [ ] Registry credentials are stored securely (e.g., GitHub Secrets, GitLab CI variables). - [ ] Build cache is enabled and effective; check pipeline logs for cache hits. - [ ] Image size is within expected range (e.g., under 200MB for a typical Node service); size reported in pipeline artifacts or logs. - [ ] Container startup is tested in pipeline via a smoke test step. - [ ] Rollback procedure is documented and tested at least once. - [ ] Regular review of Dockerfile for updates and security patches (e.g., base image updates monthly). - [ ] Monitoring and alerting for registry and deployment failures. - [ ] Non-root user configured in final stage. - [ ] Immutable tags used for production, with `latest` only for development convenience. ## Conclusion Automating Docker multi-stage builds in CI/CD streamlines the path from code to deployable artifact. By following the steps in this guide, you can create lean images, reduce build times with caching, and maintain a reliable pipeline. Start with a narrow pilot, as recommended: pick a single service, implement the Dockerfile and pipeline, verify thoroughly, and then expand to other services. Remember to continuously monitor and iterate. Assign clear ownership for the pipeline configuration and rollback procedures to avoid ambiguity. Next steps: - Apply the multi-stage Dockerfile to one of your applications. - Set up the CI/CD pipeline and push to a registry. - Run the verification commands and review the checklist. - Document and test your rollback plan. - Share the approach with your team and gather feedback. By adopting these practices, you will enhance your team's delivery efficiency and reduce production risks.