Docker Compose Capacity Planning With Practical Examples

Capacity planning in Docker Compose is the practical discipline of right-sizing CPU, memory, and I/O for each service so your stack stays fast, predictable, and recoverable as load grows. This guide walks through a complete loop: inventory your environment, apply safe resource limits in Compose file format 2.4, verify the limits are enforced, define scaling signals, handle common failure modes, and run a repeatable operations checklist. The examples use Nginx, PostgreSQL, and Redis. All numbers marked as constructed examples are starting points only; measure and refine for your workload.

---

Version and Environment Inventory

Before changing limits, build a short inventory. You need to know exactly what you are sizing and what guardrails the host supports.

Prerequisites:

• Host OS: Linux, or Docker Desktop with a Linux backend
• Docker Engine: 20.10+ (cgroups v2-capable hosts recommended)
• Docker Compose plugin: v2.20+ (invoked as docker compose)
• Sudo access for system and Docker commands

Inventory checklist:

• Record Docker Engine and Compose versions: docker version, docker compose version
• Host capacity: nproc, free -h, lsblk -o NAME,SIZE,TYPE,MOUNTPOINT
• Cgroups: mount | grep cgroup, confirm memory and pids controllers enabled
• Current Compose topology: services, networks, volumes
• Baseline metrics: idle CPU, memory footprint, disk latency (e.g., iostat -x 1 5)

Keep this inventory in your repo or runbook so future changes reference a known baseline.

---

Safe Configuration Path

For standalone Docker Compose (non-Swarm), use Compose file format 2.4 to apply per-service limits directly. Avoid deploy.resources because it is intended for Swarm scheduling and may be ignored in non-Swarm Compose.

Key options in Compose v2.4:

• cpus: fractional CPU limit per container (e.g., 0.50)
• mem_limit: hard memory cap (e.g., 512m)
• mem_reservation: soft memory reservation guidance (e.g., 384m)
• pids_limit: maximum process count per container
• ulimits: file descriptors and other per-process limits
• shm_size: shared memory size (important for PostgreSQL)
• restart: container restart policy for resiliency
• healthcheck: basic liveness checks to order dependencies and verify startup

Safety tips:

• Roll out limits incrementally. Prefer starting with reservations and gentle hard caps.
• Maintain 20–30% headroom during normal operation for CPU and memory.
• Change one variable per iteration; verify before additional changes.

---

Constructed Example: Nginx, PostgreSQL, Redis

Below is a constructed docker-compose.yml using Compose file format 2.4. Replace image versions with those you standardize on. Numbers are starting points; tune with measurements.

version: "2.4"

services:
  web:
    image: nginx:1.27-alpine
    ports:
      - "8080:80"
    depends_on:
      - cache
      - db
    healthcheck:
      test: ["CMD-SHELL", "wget -qO- http://localhost/ || exit 1"]
      interval: 10s
      timeout: 3s
      retries: 5
    cpus: "0.50"                 # constructed example starting point
    mem_limit: "256m"
    mem_reservation: "192m"
    pids_limit: 256
    ulimits:
      nofile:
        soft: 65536
        hard: 65536
    restart: unless-stopped
    networks: [appnet]

  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: app
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: appdb
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d appdb"]
      interval: 10s
      timeout: 5s
      retries: 10
    cpus: "1.00"                 # constructed example starting point
    mem_limit: "2g"
    mem_reservation: "1g"
    shm_size: "512m"             # better for query plans and sorts
    pids_limit: 4096
    ulimits:
      nofile:
        soft: 262144
        hard: 262144
    restart: unless-stopped
    networks: [appnet]

  cache:
    image: redis:7-alpine
    command: ["redis-server", "--save", "", "--appendonly", "no"]
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 3s
      retries: 10
    cpus: "0.50"                 # constructed example starting point
    mem_limit: "512m"
    mem_reservation: "384m"
    pids_limit: 1024
    ulimits:
      nofile:
        soft: 65536
        hard: 65536
    restart: unless-stopped
    networks: [appnet]

networks:
  appnet:
    driver: bridge

volumes:
  pgdata:

Why these settings (constructed rationale):

• web is I/O bound and benefits from file descriptor headroom. Modest CPU and RAM are sufficient at small scale.
• db is memory-sensitive; giving it more RAM reduces I/O by enabling larger caches. shm_size reduces temporary file spills during large sorts and hash joins.
• cache is single-threaded but latency sensitive; CPU 0.5 with 384–512 MB RAM provides a safe starting range.

Constructed starting limits summary (edit as you learn):

Source: constructed example for initial sizing only

Bring up the stack and confirm:

# Validate the file
docker compose -f docker-compose.yml config

# Start services
docker compose -f docker-compose.yml up -d

# List containers and health
docker compose ps

Expected result:

• All services show Up with (healthy) after their checks pass.
• Host CPU and memory usage remain within safe headroom.

---

Verification and Diagnostics

1. Confirm limits applied

# Replace <id> with the container ID of each service
cid_web=$(docker compose ps -q web)
cid_db=$(docker compose ps -q db)
cid_cache=$(docker compose ps -q cache)

# Check CPU and memory constraints
for c in $cid_web $cid_db $cid_cache; do
  echo "Inspecting $c";
  docker inspect $c --format='Name: {{.Name}}
 NanoCPUs: {{.HostConfig.NanoCpus}}
 Memory: {{.HostConfig.Memory}}
 PidsLimit: {{.HostConfig.PidsLimit}}
 ShmSize: {{.HostConfig.ShmSize}}
';
done

Expected result:

• NanoCpus matches your cpus multiplied by 1e9 (e.g., 0.5 → 500000000).
• Memory equals the bytes for your mem_limit (e.g., 256m → 268435456).
• PidsLimit and ShmSize (db) match your config.

2. Observe real-time usage

docker stats --no-stream

Expected result:

• MEM USAGE / LIMIT stays comfortably under LIMIT during typical activity.
• CPU % floats below your effective CPU allocation during steady state.

3. Validate health and readiness

# Health status
docker inspect $cid_web --format='{{json .State.Health}}' | jq .

# Simple HTTP smoke check
curl -s -o /dev/null -w '%{http_code}\n' http://localhost:8080/

Expected result:

• Health status reports healthy for all services.
• curl returns 200 from web.

4. Log diagnostics

docker compose logs --since=10m --tail=200

Scan for OOM, throttle, or retry messages. Typical signals include OOMKilled, connection resets, or write stalls.

5. Light load probe (constructed)

You can simulate a small burst to ensure headroom exists:

# 50 quick requests, serialized (constructed example)
for i in $(seq 1 50); do curl -s -o /dev/null -w '.' http://localhost:8080/; done; echo

Expected result:

• Responses complete quickly without error spikes or container restarts.

---

Scaling Signals and Limits

Define clear signals and thresholds that trigger scaling or resizing. The following are constructed example numbers you can adopt for pilots and refine later.

Source: constructed initial thresholds; calibrate with your SLOs

Notes on actions:

• Prefer horizontal scaling for stateless tiers; increase per-replica limits only when utilization shows CPU throttling or garbage-collection pressure.
• For PostgreSQL, extra RAM often outperforms CPU increases until the working set fits better in cache.
• Maintain a safety margin: aim for 20–30% idle capacity during steady state.

Scaling example (constructed worker):

If you have a stateless worker service without host-bound ports, you can add replicas safely:

# Constructed example: scale a worker service
# Replace 'worker' with your actual service name
docker compose up -d --scale worker=3

Expected result:

• New replicas join the same network and consume the same queue/topic.
• Host CPU and memory remain within headroom. If not, revert or add host capacity.

---

Failure Modes and Recovery

Common issues you will encounter and how to recover safely.

Source: constructed runbook entries for common containerized services

Rollback and recovery steps:

1. Immediate relief

• Temporarily raise mem_limit or cpus for the impacted service, then docker compose up -d to apply.
• If the host is exhausted, scale down non-critical services first or move load off-host.

1. Known-good configuration

• Revert to the last known-good compose file commit:

     git checkout <known_good_commit> -- docker-compose.yml
     docker compose up -d

1. Database recovery

• If PostgreSQL is in crash recovery, give it time to replay WAL. If corruption is suspected, restore from your latest verified backup to the pgdata volume.

1. Validate recovery

• Confirm healthchecks are green, docker stats is stable, and application probes succeed.

---

Operations Checklist

Use this checklist during capacity reviews and before releases that may change load patterns.

• Plan a small, inspectable pilot change and define success metrics (constructed numbers are acceptable for first pass):
• CPU ≤ 70% for steady 10-minute windows
• Memory headroom ≥ 25%
• p99 latency within your SLO
• Inventory environment and baselines:
• Record Docker/Compose versions, host CPU/RAM/disk
• Capture docker stats --no-stream for all services
• Apply limits safely (Compose v2.4):
• Set cpus, mem_limit, mem_reservation, pids_limit, and ulimits
• Add healthcheck and restart policies
• Bring up and verify:
• docker compose up -d, docker compose ps
• docker inspect to confirm NanoCPUs/Memory/PIDs
• docker stats for usage vs limits
• App smoke checks (HTTP 200, cache PING, DB ready)
• Light-load exercise:
• Run a small burst and observe docker stats, logs, and latency
• Decide on next step:
• If headroom is adequate, keep limits and document
• If constrained, adjust one limit or add one replica; repeat verification
• Document decisions:
• Record final limits, observed utilization, and rationale
• Note rollback path and backup status
• Schedule a follow-up review:
• Re-check after peak hours or major feature launches

---

Conclusion

Compose capacity planning is a repeatable loop: set guardrails, verify, observe, and adjust. Start with a narrow pilot that is easy to inspect, prefer simple and explicit limits in Compose v2.4, and keep a clear record of baseline metrics and decisions. With small, safe iterations and well-defined signals, you can scale services predictably while preserving operational safety margins. The goal is not a perfect static configuration but a disciplined process that lets you respond to real load patterns without guesswork or emergency firefighting.