Capacity planning in Docker Compose is the practical discipline of right-sizing CPU, memory, and I/O for each service so your stack stays fast, predictable, and recoverable as load grows. This guide walks through a complete loop: inventory your environment, apply safe resource limits in Compose file format 2.4, verify the limits are enforced, define scaling signals, handle common failure modes, and run a repeatable operations checklist. The examples use Nginx, PostgreSQL, and Redis. All numbers marked as constructed examples are starting points only; measure and refine for your workload. --- ## Version and Environment Inventory Before changing limits, build a short inventory. You need to know exactly what you are sizing and what guardrails the host supports. Prerequisites: - Host OS: Linux, or Docker Desktop with a Linux backend - Docker Engine: 20.10+ (cgroups v2-capable hosts recommended) - Docker Compose plugin: v2.20+ (invoked as docker compose ) - Sudo access for system and Docker commands Inventory checklist: - Record Docker Engine and Compose versions: docker version , docker compose version - Host capacity: nproc , free -h , lsblk -o NAME,SIZE,TYPE,MOUNTPOINT - Cgroups: mount | grep cgroup , confirm memory and pids controllers enabled - Current Compose topology: services, networks, volumes - Baseline metrics: idle CPU, memory footprint, disk latency (e.g., iostat -x 1 5 ) Keep this inventory in your repo or runbook so future changes reference a known baseline. --- ## Safe Configuration Path For standalone Docker Compose (non-Swarm), use Compose file format 2.4 to apply per-service limits directly. Avoid deploy.resources because it is intended for Swarm scheduling and may be ignored in non-Swarm Compose. Key options in Compose v2.4: - cpus : fractional CPU limit per container (e.g., 0.50 ) - mem_limit : hard memory cap (e.g., 512m ) - mem_reservation : soft memory reservation guidance (e.g., 384m ) - pids_limit : maximum process count per container - ulimits : file descriptors and other per-process limits - shm_size : shared memory size (important for PostgreSQL) - restart : container restart policy for resiliency - healthcheck : basic liveness checks to order dependencies and verify startup Safety tips: - Roll out limits incrementally. Prefer starting with reservations and gentle hard caps. - Maintain 20–30% headroom during normal operation for CPU and memory. - Change one variable per iteration; verify before additional changes. --- ## Constructed Example: Nginx, PostgreSQL, Redis Below is a constructed docker-compose.yml using Compose file format 2.4. Replace image versions with those you standardize on. Numbers are starting points; tune with measurements. version: "2.4" services: web: image: nginx:1.27-alpine ports: - "8080:80" depends_on: - cache - db healthcheck: test: ["CMD-SHELL", "wget -qO- http://localhost/ || exit 1"] interval: 10s timeout: 3s retries: 5 cpus: "0.50" # constructed example starting point mem_limit: "256m" mem_reservation: "192m" pids_limit: 256 ulimits: nofile: soft: 65536 hard: 65536 restart: unless-stopped networks: [appnet] db: image: postgres:16-alpine environment: POSTGRES_USER: app POSTGRES_PASSWORD: secret POSTGRES_DB: appdb volumes: - pgdata:/var/lib/postgresql/data healthcheck: test: ["CMD-SHELL", "pg_isready -U app -d appdb"] interval: 10s timeout: 5s retries: 10 cpus: "1.00" # constructed example starting point mem_limit: "2g" mem_reservation: "1g" shm_size: "512m" # better for query plans and sorts pids_limit: 4096 ulimits: nofile: soft: 262144 hard: 262144 restart: unless-stopped networks: [appnet] cache: image: redis:7-alpine command: ["redis-server", "--save", "", "--appendonly", "no"] healthcheck: test: ["CMD", "redis-cli", "ping"] interval: 10s timeout: 3s retries: 10 cpus: "0.50" # constructed example starting point mem_limit: "512m" mem_reservation: "384m" pids_limit: 1024 ulimits: nofile: soft: 65536 hard: 65536 restart: unless-stopped networks: [appnet] networks: appnet: driver: bridge volumes: pgdata: Why these settings (constructed rationale): - web is I/O bound and benefits from file descriptor headroom. Modest CPU and RAM are sufficient at small scale. - db is memory-sensitive; giving it more RAM reduces I/O by enabling larger caches. shm_size reduces temporary file spills during large sorts and hash joins. - cache is single-threaded but latency sensitive; CPU 0.5 with 384–512 MB RAM provides a safe starting range. Constructed starting limits summary (edit as you learn):
ServiceCPU (cpus)Mem limitMem reservationPIDs limitNotes
web0.50256m192m256Increase nofile for concurrent connections
db1.002g1g4096Larger RAM reduces disk I/O; add shm_size
cache0.50512m384m1024Keep AOF off for latency-sensitive caches
Source: constructed example for initial sizing only Bring up the stack and confirm: # Validate the file docker compose -f docker-compose.yml config # Start services docker compose -f docker-compose.yml up -d # List containers and health docker compose ps Expected result: - All services show Up with (healthy) after their checks pass. - Host CPU and memory usage remain within safe headroom. --- ## Verification and Diagnostics ### 1. Confirm limits applied # Replace with the container ID of each service cid_web=$(docker compose ps -q web) cid_db=$(docker compose ps -q db) cid_cache=$(docker compose ps -q cache) # Check CPU and memory constraints for c in $cid_web $cid_db $cid_cache; do echo "Inspecting $c"; docker inspect $c --format='Name: {{.Name}} NanoCPUs: {{.HostConfig.NanoCpus}} Memory: {{.HostConfig.Memory}} PidsLimit: {{.HostConfig.PidsLimit}} ShmSize: {{.HostConfig.ShmSize}} '; done Expected result: - NanoCpus matches your cpus multiplied by 1e9 (e.g., 0.5 → 500000000). - Memory equals the bytes for your mem_limit (e.g., 256m → 268435456). - PidsLimit and ShmSize (db) match your config. ### 2. Observe real-time usage docker stats --no-stream Expected result: - MEM USAGE / LIMIT stays comfortably under LIMIT during typical activity. - CPU % floats below your effective CPU allocation during steady state. ### 3. Validate health and readiness # Health status docker inspect $cid_web --format='{{json .State.Health}}' | jq . # Simple HTTP smoke check curl -s -o /dev/null -w '%{http_code}\n' http://localhost:8080/ Expected result: - Health status reports healthy for all services. - curl returns 200 from web. ### 4. Log diagnostics docker compose logs --since=10m --tail=200 Scan for OOM, throttle, or retry messages. Typical signals include OOMKilled , connection resets, or write stalls. ### 5. Light load probe (constructed) You can simulate a small burst to ensure headroom exists: # 50 quick requests, serialized (constructed example) for i in $(seq 1 50); do curl -s -o /dev/null -w '.' http://localhost:8080/; done; echo Expected result: - Responses complete quickly without error spikes or container restarts. --- ## Scaling Signals and Limits Define clear signals and thresholds that trigger scaling or resizing. The following are constructed example numbers you can adopt for pilots and refine later.
SignalTarget window (constructed)ActionHow to check
CPU utilization≤ 70% for 10+ minutesAdd 0.5 CPU or add 1 replica (stateless)docker stats, app APM
Memory headroom≥ 25% free vs limitIncrease mem_limit by 25–50%docker stats, OOM logs
p99 HTTP latency≤ 300 ms steadyScale web/app replicas; tune keepaliveApp metrics, access logs
Redis ops latency≤ 2 ms p99Add CPU or reduce AOF/durabilityRedis INFO, app metrics
DB checkpoint time≤ 1 minIncrease RAM, tune WAL/checkpointPostgres logs, pg_stat_bgwriter
Queue depthDrain within SLAScale workers horizontallyApp/queue metrics
Source: constructed initial thresholds; calibrate with your SLOs Notes on actions: - Prefer horizontal scaling for stateless tiers; increase per-replica limits only when utilization shows CPU throttling or garbage-collection pressure. - For PostgreSQL, extra RAM often outperforms CPU increases until the working set fits better in cache. - Maintain a safety margin: aim for 20–30% idle capacity during steady state. Scaling example (constructed worker): If you have a stateless worker service without host-bound ports, you can add replicas safely: # Constructed example: scale a worker service # Replace 'worker' with your actual service name docker compose up -d --scale worker=3 Expected result: - New replicas join the same network and consume the same queue/topic. - Host CPU and memory remain within headroom. If not, revert or add host capacity. --- ## Failure Modes and Recovery Common issues you will encounter and how to recover safely.
Failure modeSymptomQuick checkRecovery
OOM killContainer restarts, OOMKilled=truedocker inspect .State.OOMKilled, logsIncrease mem_limit, reduce heap/cache, or split workload; restart
CPU throttlingHigh latency, uneven throughputdocker stats high CPU %, dmesg CFS throttlingIncrease cpus, scale out, or reduce per-request cost
I/O saturationHigh iowait, slow DB writesiostat -x, Postgres slow checkpointsIncrease RAM (cache), move data to faster disk, tune WAL/checkpoint
File descriptor exhaustionEMFILE errorsApp logs, ulimit -n in containerIncrease ulimits.nofile; restart
PIDs limit reachedFork/exec failuresdocker inspect .HostConfig.PidsLimitRaise pids_limit; review process model
Log growth fills diskDisk 100% fulldf -h, log sizeRotate/truncate logs, move to external logging, cap retention
Source: constructed runbook entries for common containerized services Rollback and recovery steps: - Immediate relief - Temporarily raise mem_limit or cpus for the impacted service, then docker compose up -d to apply. - If the host is exhausted, scale down non-critical services first or move load off-host. - Known-good configuration - Revert to the last known-good compose file commit: git checkout -- docker-compose.yml docker compose up -d - Database recovery - If PostgreSQL is in crash recovery, give it time to replay WAL. If corruption is suspected, restore from your latest verified backup to the pgdata volume. - Validate recovery - Confirm healthchecks are green, docker stats is stable, and application probes succeed. --- ## Operations Checklist Use this checklist during capacity reviews and before releases that may change load patterns. - Plan a small, inspectable pilot change and define success metrics (constructed numbers are acceptable for first pass): - CPU ≤ 70% for steady 10-minute windows - Memory headroom ≥ 25% - p99 latency within your SLO - Inventory environment and baselines: - Record Docker/Compose versions, host CPU/RAM/disk - Capture docker stats --no-stream for all services - Apply limits safely (Compose v2.4): - Set cpus , mem_limit , mem_reservation , pids_limit , and ulimits - Add healthcheck and restart policies - Bring up and verify: - docker compose up -d , docker compose ps - docker inspect to confirm NanoCPUs / Memory / PIDs - docker stats for usage vs limits - App smoke checks (HTTP 200, cache PING, DB ready) - Light-load exercise: - Run a small burst and observe docker stats , logs, and latency - Decide on next step: - If headroom is adequate, keep limits and document - If constrained, adjust one limit or add one replica; repeat verification - Document decisions: - Record final limits, observed utilization, and rationale - Note rollback path and backup status - Schedule a follow-up review: - Re-check after peak hours or major feature launches --- ## Conclusion Compose capacity planning is a repeatable loop: set guardrails, verify, observe, and adjust. Start with a narrow pilot that is easy to inspect, prefer simple and explicit limits in Compose v2.4, and keep a clear record of baseline metrics and decisions. With small, safe iterations and well-defined signals, you can scale services predictably while preserving operational safety margins. The goal is not a perfect static configuration but a disciplined process that lets you respond to real load patterns without guesswork or emergency firefighting.