Docker Compose Architecture Explained with Practical Examples

Intro

Docker Compose is the de facto tool for defining and running multi-container applications on a single Docker host. While many developers can write a basic docker-compose.yml, fewer understand the architecture underneath: how the Compose CLI translates declarative YAML into container runtime commands, how services discover each other, how volumes and networks are managed, and where failures are most likely to occur.

This guide explains Docker Compose architecture from an operational perspective. You will learn the core components, the flow of data and control during up, down, and build, and the practical tradeoffs that affect reliability and maintainability. We will walk through a concrete example: a web application with a backend API, a database, and a reverse proxy. By the end, you will be able to diagnose common Compose issues, roll back safely, and operate multi-service environments with confidence.

A well-structured Docker Compose architecture separates concerns between the CLI, the Compose file model, and the underlying container runtime. Understanding these components reduces operational rework and makes configuration, networking, and storage explicit.

Version and Environment Inventory

Before changing any Compose configuration, record the versions and topology of your environment. This inventory is your baseline for troubleshooting and rollback.

Docker and Compose versions

Docker Compose exists in two forms: the standalone docker-compose binary (v1) and the integrated docker compose plugin (v2). As of 2024, v1 is deprecated and v2 is the default in Docker Desktop and Docker Engine installations. Check your versions with:

docker --version
docker compose version

Expected output (example):

Docker version 24.0.7, build afdd53b
Docker Compose version v2.23.0-desktop.1

Only v2 supports features like docker compose watch, --profile, and the newer depends_on conditions. If you are still on v1, upgrade before proceeding.

Current topology

Document the existing services and their relationships. For an existing Compose project, run:

docker compose ps

Example output:

NAME                IMAGE               COMMAND                  SERVICE   CREATED         STATUS         PORTS
myapp_web_1         myapp_web:latest    "nginx -g 'daemon of..." web       2 hours ago     Up 2 hours     0.0.0.0:80->80/tcp
myapp_api_1         myapp_api:latest    "uvicorn app.main: ap..." api      2 hours ago     Up 2 hours     8000/tcp
myapp_db_1          postgres:15         "docker-entrypoint.s..." db       2 hours ago     Up 2 hours     5432/tcp

This shows container names, images, commands, and port mappings. Note that in a default Compose network, services can reach each other by service name (api, db), not by container name.

Prerequisites

• Docker Engine 20.10+ with the Compose v2 plugin.
• At least 4 GB RAM and 10 GB disk space for a typical three-service stack.
• A directory for the project, e.g., ~/myapp, with write permissions.
• Basic familiarity with Dockerfiles and command-line Docker.

Safe Configuration Path

This section builds a minimal but realistic Compose configuration step by step. We will create a web frontend (nginx), an API service (Python/FastAPI), and a PostgreSQL database. The configuration choices are scoped to be easy to inspect and roll back.

Directory structure and initial files

Create the project directory:

mkdir -p ~/myapp/nginx ~/myapp/api
cd ~/myapp

Create api/app.py:

from fastapi import FastAPI
import psycopg2
import os

app = FastAPI()

@app.get("/")
def read_root():
    return {"Hello": "World"}

@app.get("/db")
def read_db():
    conn = psycopg2.connect(
        host=os.environ.get("DB_HOST", "db"),
        database=os.environ.get("DB_NAME", "mydb"),
        user=os.environ.get("DB_USER", "user"),
        password=os.environ.get("DB_PASSWORD", "password")
    )
    cur = conn.cursor()
    cur.execute("SELECT version();")
    version = cur.fetchone()[0]
    cur.close()
    conn.close()
    return {"db_version": version}

Create api/requirements.txt:

fastapi
uvicorn
psycopg2-binary

Create api/Dockerfile:

FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app.py .
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

Create nginx/nginx.conf:

events {}
http {
    upstream api {
        server api:8000;
    }
    server {
        listen 80;
        location / {
            proxy_pass http://api;
        }
    }
}

Create nginx/Dockerfile:

FROM nginx:1.25-alpine
COPY nginx.conf /etc/nginx/nginx.conf

Compose file: initial version

Create docker-compose.yml:

version: "3.8"
services:
  db:
    image: postgres:15
    environment:
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password
      POSTGRES_DB: mydb
    volumes:
      - db-data:/var/lib/postgresql/data
  api:
    build: ./api
    environment:
      DB_HOST: db
      DB_NAME: mydb
      DB_USER: user
      DB_PASSWORD: password
    depends_on:
      - db
  web:
    build: ./nginx
    ports:
      - "80:80"
    depends_on:
      - api
volumes:
  db-data:

This configuration defines three services, a named volume for database persistence, and an implicit default network. The version key is optional in Compose v2 but often included for compatibility.

Explanation of architecture components

• Compose file model: The YAML file is parsed into a project model. Services map to containers; networks and volumes are first-class resources.
• Project name: By default, the project name is the directory name (myapp). All resources are prefixed with this name to avoid collisions.
• Networks: Compose creates a single default network (e.g., myapp_default) and attaches all services. Each service gets a DNS entry matching its service name. This enables service discovery without hardcoding IPs.
• Volumes: Named volumes (db-data) persist data independently of container lifecycle.
• Dependencies: depends_on controls startup order but does not guarantee readiness. We will address this later.

Build and start

Run in the foreground for first inspection:

docker compose up --build

Expected output (truncated):

[+] Building 12.3s (12/12) FINISHED
...
[+] Running 3/3
 ✔ Container myapp-db-1 Created
 ✔ Container myapp-api-1 Created
 ✔ Container myapp-web-1 Created
Attaching to myapp-db-1, myapp-api-1, myapp-web-1
...

Press Ctrl+C to stop. Later, start in detached mode:

docker compose up -d

Scoped implementation choices

• No custom networks: Using the default network is sufficient for a single-host setup. Custom networks become necessary for isolation or custom subnets.
• No healthchecks yet: We add them later after verifying the basic flow.
• Plain text secrets: For local development only. In production, use Docker secrets or environment variable files with restricted permissions.
• Persistent volume for db: Ensures data survives docker compose down (without -v).

Verification and Diagnostics

After starting the stack, verify each component and the overall data flow.

Service status

docker compose ps

All services should show Up status and no restart loops.

Network connectivity

Check that the web container can reach the API by service name:

docker compose exec web wget -qO- http://api:8000/

Expected output:

{"Hello":"World"}

If this fails, check the API container logs:

docker compose logs api

Database connectivity

Verify the API can query the database:

docker compose exec api python -c "import psycopg2; conn = psycopg2.connect(host='db', database='mydb', user='user', password='password'); cur = conn.cursor(); cur.execute('SELECT version()'); print(cur.fetchone()[0]); conn.close()"

Expected output (truncated):

PostgreSQL 15.4 ...

HTTP end-to-end

Access the web service from the host:

curl http://localhost/

Expected output:

{"Hello":"World"}

And the database endpoint:

curl http://localhost/db

Expected output (truncated):

{"db_version":"PostgreSQL 15.4 ..."}

Inspect the network

List networks and inspect the default network:

docker network ls

Find myapp_default. Inspect it:

docker network inspect myapp_default

This shows the containers attached with their IPs and aliases. Each service has an alias matching its service name.

Check volume mapping

Inspect the volume:

docker volume inspect myapp_db-data

Confirm the mountpoint exists and is owned by the postgres container.

Healthchecks and readiness

Currently, depends_on only waits for container start, not application readiness. The API may fail to connect to the database if the database is still initializing. Add healthchecks to the Compose file:

services:
  db:
    image: postgres:15
    # ... existing config
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U user -d mydb"]
      interval: 5s
      timeout: 5s
      retries: 5
  api:
    # ... existing config
    depends_on:
      db:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/"]
      interval: 10s
      timeout: 5s
      retries: 3
  web:
    # ... existing config
    depends_on:
      api:
        condition: service_healthy

Apply changes:

docker compose up -d

Now Compose waits for healthy dependencies before starting dependents. Verify with docker compose ps - the STATUS column should show (healthy) for db and api.

Verification checklist

Failure Modes and Recovery

Understanding common failure modes allows quick recovery and rollback.

1. Service fails to start due to missing dependency

Symptom: API container restarts repeatedly with error could not connect to server: Connection refused. Cause: Database not ready when API tries to connect, or database credentials mismatch. Diagnosis: Check logs:

docker compose logs api

Recovery: Ensure healthchecks are defined. If credentials mismatch, fix environment variables and recreate services:

docker compose up -d --force-recreate

2. Port already in use on host

Symptom: web service fails with Bind for 0.0.0.0:80 failed: port is already allocated. Cause: Another process uses port 80. Diagnosis: Find the process:

sudo lsof -i :80

Recovery: Change host port in Compose file to e.g., 8080:80, then docker compose up -d.

3. Volume data corruption or unintended deletion

Symptom: Database loses data after docker compose down -v. Cause: The -v flag removes named volumes. Recovery: Restore from backup if available. To prevent, avoid -v unless intentionally wiping. Back up the volume periodically:

docker run --rm -v myapp_db-data:/data -v $(pwd):/backup alpine tar czf /backup/db-backup.tar.gz -C /data .

4. Configuration change causes service breakage

Symptom: After editing docker-compose.yml or Dockerfile, services fail. Recovery: Roll back to previous working configuration:

• Keep Compose files in version control.
• Revert the file and run docker compose up -d --build.
• Compose will recreate only changed services.

5. Network partition or DNS failure inside Compose network

Symptom: Services cannot resolve each other's names. Diagnosis: Exec into a container and ping another service:

docker compose exec web ping api

If ping fails, check network:

docker network inspect myapp_default

Ensure all containers are attached. If not, recreate the network:

docker compose down
# Remove the network manually if needed
docker network rm myapp_default
docker compose up -d

6. Disk space exhaustion from images and volumes

Symptom: Build fails with no space left on device. Recovery: Clean up unused resources:

docker system prune -a --volumes

Use with caution; this removes all unused containers, networks, images, and volumes.

7. Unexpected container exit

Symptom: Container exits with code 137 or 143. Cause: Out-of-memory killer or graceful shutdown signal. Diagnosis: Check container exit code:

docker inspect myapp-api-1 --format='{{.State.ExitCode}}'

Recovery: Increase memory limits in Compose (mem_limit) or adjust application memory usage.

Common failure modes and recovery actions

Operations Checklist

Use this checklist for routine operation and review of a Docker Compose environment.

Daily operation

• Run docker compose ps to check all services are healthy.
• Check logs for errors: docker compose logs --tail=100.
• Monitor disk usage: docker system df.
• Verify end-to-end functionality: curl http://localhost/.

Weekly maintenance

• Back up database volume (or automate via cron).
• Review resource usage: docker stats --no-stream.
• Check for image updates: docker compose pull (then test before applying).
• Inspect network for unexpected attachments.

Before configuration changes

• Commit current working Compose files to version control.
• Document the change and expected impact.
• Test changes in a staging environment if possible.

After configuration changes

• Run docker compose config to validate YAML syntax and resolved variables.
• Apply changes with docker compose up -d.
• Run verification checklist (Section 4).
• Monitor logs for 10 minutes for unexpected errors.

Rollback plan

• Keep previous Compose file version in VCS.
• If rollback needed, checkout previous file and run docker compose up -d --build.
• If volume schema changed, ensure backup is restorable.

Security review (quarterly)

• Review exposed ports; minimize host port bindings.
• Rotate database passwords and update environment variables.
• Scan images for vulnerabilities: docker scan or third-party tools.
• Ensure Compose files do not contain secrets in plain text; use .env files or Docker secrets.

Conclusion

Docker Compose architecture is more than a convenience layer; it is a structured model for multi-container applications. By understanding the components - CLI, project model, networks, volumes, and lifecycle management - you can design reliable local and single-host deployments.

In this guide, you learned:

• The core components: Compose file, services, networks, volumes, and the project name.
• How to construct a safe configuration with explicit dependencies and healthchecks.
• How to verify the stack with concrete diagnostic commands and expected outputs.
• Common failure modes and their recovery procedures.
• An operations checklist for daily, weekly, and change management tasks.

For a first practical pilot, start with a narrow, measurable multi-service setup that can be inspected locally before expanding. Record versions and topology, build incrementally, verify each layer, and always have a rollback plan. This approach reduces operational rework and builds confidence for scaling to more complex environments.

Next steps:

• Implement the example stack on your machine.
• Add healthchecks and practice failure recovery.
• Back up the database volume and test restoration.
• Extend the architecture with a custom network or additional service.
• Integrate the operations checklist into your team's workflow.

With this foundation, you can operate Docker Compose environments with clarity and control.