## Intro

Advanced Docker Compose is not about running more containers. It is about controlling how services are built, started, discovered, configured, secured, and observed as a single, predictable unit. In this guide you will:

- Understand how Compose groups resources and resolves configuration so you can predict behavior.

- Apply profiles, healthchecks with conditional dependencies, secrets and configs, anchors, and multi-file overrides.

- Harden containers (read-only FS, no-new-privileges, ulimits) and avoid common misconfigurations.

- Run a focused local pilot that proves value without risking production.

## Workflow overview

Use this safe rollout flow:

- Design the service graph

- Define services, their data boundaries, and published ports. Use named volumes for state.

- Keep the default project network for DNS-based service discovery; avoid custom bridges unless necessary.

- Build

- Use a tight build context, multi-stage targets, and build args. Enable BuildKit. Prefer cached layers and a robust .dockerignore.

- Run

- Start with an explicit project name, enable optional tools via profiles, and add healthchecks. Use depends_on with conditions to gate readiness, not just start order.

- Observe

- Stream logs, inspect health, and test DNS names on the project network.

- Harden

- Add read-only filesystems where possible, set ulimits, and mount secrets as files. Avoid deploy.* unless you use Swarm.

## Compose internals that matter

- Project name

- Compose groups everything by a project name. Set it with the top-level name: field or -p flag. Networks and volumes get this prefix, isolating projects on the same host.

- Default networking

- Compose creates one user-defined bridge network per project. Each service name is a DNS hostname on that network (for example, db resolves to the Postgres container). You do not need links .

- File merging

- Multiple -f files merge top-to-bottom. Later files override or extend earlier ones. Keep a stable base file and overlay environment-specific changes.

- Environment variable resolution

- Compose supports ${VAR} and ${VAR:-default} in YAML. It reads a .env in the project directory and your shell env; shell values override .env . env_file under a service injects variables into the container only and does not affect Compose-level substitution.

- Startup ordering vs readiness

- depends_on without conditions orders start only. For readiness, define a healthcheck and use depends_on: condition: service_healthy .

- Health status

- Services with healthchecks report starting , healthy , or unhealthy . Query with docker compose ps and docker compose logs .

- Swarm-specific keys

- Keys under deploy.* are for Swarm mode. Docker Compose (single host) ignores most of them. Use service-level options ( environment , volumes , ulimits , read_only , mem_limit , cpus , etc.) for non-Swarm usage.

- Version field

- Modern Compose does not require version: . Omit it and rely on the Compose Specification supported by your CLI.

## Advanced concepts with a production-shaped example

A concise base file showing practical patterns: profiles, health-gated dependencies, secrets/configs, anchors, and safe defaults.

# compose.yml
name: myapp

# Extension fields and anchors for reuse
x-service-defaults: &service_defaults
 restart: unless-stopped
 networks:
 - appnet
 env_file:
 - ./.env
 init: true
 logging:
 driver: json-file
 options:
 max-size: "10m"
 max-file: "3"
 mem_limit: 512m
 cpus: "0.50"

x-healthchecks:
 postgres: &pg_health
 test: ["CMD-SHELL", "pg_isready -U $POSTGRES_USER -d $POSTGRES_DB"]
 interval: 3s
 timeout: 3s
 retries: 10
 start_period: 10s
 redis: &redis_health
 test: ["CMD", "redis-cli", "ping"]
 interval: 3s
 timeout: 3s
 retries: 10
 start_period: 5s
 web: &web_health
 test: ["CMD", "wget", "-qO-", "http://localhost/healthz"]
 interval: 5s
 timeout: 3s
 retries: 10
 start_period: 5s

services:
 db:
 <<: *service_defaults
 image: postgres:16
 environment:
 POSTGRES_DB: app
 POSTGRES_USER: app
 POSTGRES_PASSWORD_FILE: /run/secrets/db_password
 volumes:
 - pgdata:/var/lib/postgresql/data
 healthcheck: *pg_health
 secrets:
 - db_password
 ulimits:
 nofile: 65535

 cache:
 <<: *service_defaults
 image: redis:7
 command: ["redis-server", "--appendonly", "yes"]
 volumes:
 - redisdata:/data
 healthcheck: *redis_health

 web:
 <<: *service_defaults
 build:
 context: ./web
 dockerfile: Dockerfile
 target: runtime
 args:
 APP_ENV: ${APP_ENV:-dev}
 ssh:
 - default
 depends_on:
 db:
 condition: service_healthy
 cache:
 condition: service_healthy
 ports:
 - "127.0.0.1:8080:80" # bind locally to avoid exposing to the LAN
 healthcheck: *web_health
 read_only: true
 tmpfs:
 - /tmp
 security_opt:
 - no-new-privileges:true
 configs:
 - source: nginx_conf
 target: /etc/nginx/conf.d/app.conf

 adminer:
 <<: *service_defaults
 image: adminer:4
 profiles: ["tools"]
 ports:
 - "127.0.0.1:8081:8080"

configs:
 nginx_conf:
 file: ./deploy/nginx/app.conf

secrets:
 db_password:
 file: ./secrets/db_password.txt

networks:
 appnet: {}

volumes:
 pgdata: {}
 redisdata: {} 
 Key points:

- name: myapp sets a stable project prefix for networks and volumes.

- Anchors reduce duplication and prevent configuration drift.

- Healthchecks plus depends_on: condition: service_healthy gate the web service on Postgres and Redis readiness.

- secrets and configs mount files into containers. Protect the source files in your working directory.

- read_only and no-new-privileges harden the web container. tmpfs for /tmp avoids image writes.

- Bind ports to 127.0.0.1 for local dev to prevent unintended exposure.

## Local overrides for iterative work

Use an override to bind-mount source and enable debug flags without touching the base file. compose.override.yml is auto-loaded by docker compose .

# compose.override.yml
services:
 web:
 volumes:
 - ./web/src:/var/www/src:rw,cached
 environment:
 DEBUG: "1" 
 For explicit, environment-specific overlays, create a dedicated file and pass it with -f .

# compose.prod.yml
services:
 web:
 environment:
 APP_ENV: prod
 ports:
 - "80:80" # publish on standard port in prod
 adminer:
 profiles: ["never"] # ensure tools stay off in prod 
 Run with:

# Base + explicit prod overlay
docker compose -f compose.yml -f compose.prod.yml up -d 

## Environment variables and .env

 Create a .env in the project directory for Compose-level substitution and common values:

# .env
APP_ENV=dev
POSTGRES_PASSWORD=unused_when_using_secret
# You still need secrets/db_password.txt for the secret file 
 Notes:

- Shell variables override .env for substitution: APP_ENV=staging docker compose up uses staging .

- env_file injects variables into that container only and does not influence Compose interpolation.

- Prefer secrets for sensitive values; avoid committing them to VCS.

## Multi-stage builds with BuildKit and SSH

Use multi-stage builds to keep runtime images small. With BuildKit you can securely forward your SSH agent to fetch private deps.

Example: a compact Go service image.

# api/Dockerfile
FROM golang:1.22 AS build
WORKDIR /src
# optional: use BuildKit SSH to fetch private deps
# RUN --mount=type=ssh go env -w GOPRIVATE=github.com/yourorg/*
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod go mod download
COPY . .
RUN CGO_ENABLED=0 go build -ldflags='-s -w' -o /out/api ./cmd/api

FROM gcr.io/distroless/base-debian12 AS runtime
COPY --from=build /out/api /usr/local/bin/api
USER 65532:65532
EXPOSE 8080
ENTRYPOINT ["/usr/local/bin/api"] 
 Enable BuildKit and build with SSH forwarding:

export DOCKER_BUILDKIT=1
export COMPOSE_DOCKER_CLI_BUILD=1
ssh-add -l >/dev/null 2>&1 || ssh-add
# If the api service is in your Compose file, you can build it like this:
docker compose build --ssh default 

## Local pilot plan

 Start with a narrow, measurable pilot that exercises the most valuable features without overcomplication.

Pilot goals:

- Prove health-gated startup works (web waits for db and cache).

- Validate secrets and configs mounts.

- Confirm project scoping and service discovery across the app network.

Preparation:

# Create directories and files
mkdir -p deploy/nginx secrets web/src
printf 'server {\n listen 80;\n location /healthz { return 200 "ok"; }\n location / { return 200 "hello"; }\n}\n' > deploy/nginx/app.conf
umask 077 && printf 'supersecretpassword\n' > secrets/db_password.txt

# Optional: add .env with APP_ENV=dev 
 Run the pilot:

# Bring up base stack
docker compose up -d --build
# Or include the optional tools profile
docker compose --profile tools up -d --build 
 Validate:

# Check service states and health
docker compose ps

# Wait for all services to be healthy (supported on recent versions)
docker compose up --wait

# Hit the web endpoint
curl -sS http://localhost:8080/healthz

# Confirm DNS based discovery from web to db
docker compose exec web getent hosts db

# Verify secrets are mounted inside db
docker compose exec db ls -l /run/secrets 
 Tear down cleanly when done:

docker compose down --volumes 

## Common misconfigurations and quick fixes

- Using the latest tag everywhere

- Fix: pin images (for example, postgres:16 , redis:7 ) to avoid surprise upgrades.

- Publishing ports to the world by default

- Fix: bind to localhost during dev ( 127.0.0.1:8080:80 ). Only publish public ports in production overlays.

- Expecting depends_on to wait for readiness

- Fix: add healthcheck and depends_on: condition: service_healthy . Ensure the health endpoint is cheap and reliable.

- Confusing .env with env_file

- Fix: Use .env for Compose interpolation and env_file per-service for container env. Remember shell env overrides .env .

- Mounting database data with bind mounts

- Fix: use named volumes for databases ( pgdata , redisdata ). They are portable and safer.

- Sprinkling deploy.* options into non-Swarm Compose

- Fix: remove deploy.* . For local Compose, use mem_limit , cpus , ulimits , read_only , etc.

- Forcing container_name

- Fix: avoid container_name . It breaks scaling and name uniqueness. Let Compose generate names with the project prefix.

- Flaky healthchecks

- Fix: tune start_period , interval , and retries . Prefer explicit shells ( CMD-SHELL ) when you need shell features; otherwise use CMD arrays.

- Big, slow builds

- Fix: shrink the build context, use .dockerignore , multi-stage builds, and BuildKit cache mounts.

## Practical tips and safe defaults

- Use named volumes for databases; avoid bind-mounting database directories.

- Keep the default project network and rely on service names for DNS. Publish only the ports you truly need.

- Prefer healthchecks plus conditional depends_on over ad hoc wait scripts.

- Use profiles to keep optional tools off by default.

- Harden with read_only , no-new-privileges , ulimits , and non-root users where practical.

- Treat local secrets as sensitive: restrict permissions on ./secrets and never commit them.

- Keep Compose files small and factor overrides by environment.

## Conclusion

 You now have a practical, up-to-date model of how Docker Compose groups services, wires networking and discovery, resolves configuration, and enforces readiness. Start with the local pilot to validate health-gated startup, secrets, and configs. From there, layer on environment-specific overlays, harden containers with read-only filesystems and no-new-privileges, pin images, and keep optional tools behind profiles. This staged, disciplined approach reduces risk while unlocking Compose for fast, confident day-to-day operations.