Docker Compose Production Deployment Capacity Planning: A Practical Guide

Learn how to estimate resources, set limits, plan scaling, and manage growth risks for Docker Compose production deployments with concrete commands, verification steps, and rollback strategies.

Intro

Docker Compose is a common way to define and run multi-container applications. In development, it works well with default settings. In production, however, default settings can lead to resource exhaustion, performance degradation, and outages. Capacity planning is the process of estimating the computing resources your application needs under expected and peak loads, and configuring Docker Compose to enforce those limits safely.

This article provides practical guidance for capacity planning with Docker Compose in production. It covers environment inventory, safe configuration of resource limits, verification methods, failure modes, recovery procedures, an operations checklist, and common pitfalls. By following these steps, you can avoid common pitfalls and ensure your deployment is both efficient and resilient.

Capacity planning is not a one-time task. It must be revisited as your application evolves, traffic patterns change, and infrastructure updates. This guide gives you the tools to plan systematically and adjust confidently.

Version and Environment Inventory

Before setting resource limits, you need a clear picture of your environment. This includes Docker and Docker Compose versions, host operating system, hardware resources, and application architecture.

Run the following commands to gather versions:

docker version --format '{{.Server.Version}}'
docker compose version

Expected output example:

Docker version 24.0.5, build ced0996
docker compose version v2.20.2

Note that Docker Compose V2 is a plugin, and the command is docker compose (without hyphen). Older V1 uses docker-compose. This guide assumes V2.

Check host resources with:

free -h
nproc

Example output:

              total        used        free      shared  buff/cache   available
Mem:           15Gi       2.1Gi        12Gi       0.0Ki       1.2Gi        13Gi
Swap:         2.0Gi          0B       2.0Gi
4

This shows 4 CPUs and 15 GiB RAM. All resource planning must fit within these host limits, leaving headroom for the operating system and Docker itself. A common rule is to reserve at least 20% of total memory and one full CPU for the host system and Docker daemon overhead.

Inspect your current containers, if any:

docker ps --format 'table {{.Names}}\t{{.Image}}\t{{.Status}}'

List images and their sizes:

docker images --format 'table {{.Repository}}:{{.Tag}}\t{{.Size}}'

Document the topology: which services run on which ports, their dependencies, and any external systems. This inventory will guide your capacity calculations.

For example, a typical web application stack might have:

Also check the host's disk space and I/O capabilities, since container logs and database storage can consume significant resources.

df -h
iostat -x 1 3

Safe Configuration Path

With the environment inventory complete, you can set safe resource limits in your Compose file. Resource limits prevent any single container from consuming all host resources and causing system instability. Use the deploy.resources section, which is respected by Docker Compose V2 when deploying to a single node.

For each service, define limits (maximum allowed) and reservations (guaranteed minimum). A safe starting point is to limit each service to a fraction of total host resources. For example, on a host with 4 CPUs and 16 GB RAM, you might allocate:

• Web service: 1 CPU, 512 MB RAM
• Database: 2 CPUs, 4 GB RAM
• Cache: 0.5 CPU, 256 MB RAM

Example Compose snippet:

services:
  web:
    image: nginx:1.25
    deploy:
      resources:
        limits:
          cpus: '1.0'
          memory: 512M
        reservations:
          cpus: '0.5'
          memory: 256M
  db:
    image: postgres:15
    deploy:
      resources:
        limits:
          cpus: '2.0'
          memory: 4G
        reservations:
          cpus: '1.0'
          memory: 2G

Note: cpus is a string value with quotes. Memory can be specified in bytes, kilobytes (K), megabytes (M), or gigabytes (G).

To apply, use docker compose up -d after modifying the file. Then verify limits are set:

docker inspect web --format '{{.HostConfig.NanoCpus}} {{.HostConfig.Memory}}'

Expected output (NanoCpus is in 1e-9 CPU units):

1000000000 536870912

This confirms 1 CPU and 512 MB limit. Adjust limits based on measured usage (see Verification section).

When setting limits, consider the following:

• Leave headroom: Ensure the sum of all service limits does not exceed 80% of host resources.
• Reservations vs. limits: Reservations guarantee a minimum, which can improve scheduling in Swarm but are informational on a single node.
• Application knowledge: Databases often need more memory; stateless web servers can run with less.

Verification and Diagnostics

After applying resource limits, verify they are effective and that the application performs as expected. Use Docker's built-in statistics commands to observe resource consumption.

Monitor live usage:

docker stats --no-stream

Example output:

CONTAINER ID   NAME      CPU %     MEM USAGE / LIMIT     MEM %     NET I/O           BLOCK I/O         PIDS
abcd1234efgh   web       0.50%     45MiB / 512MiB        8.79%     1.2kB / 2.3kB     0B / 0B           5
ijkl5678mnop   db        2.10%     1.2GiB / 4GiB         30.00%    5.4kB / 6.7kB     10MB / 20MB       27

Key metrics: CPU % relative to allocated CPUs, memory usage vs limit, and I/O. If a container consistently hits its memory limit, it may be killed by the OOM killer. To check for OOM events, inspect the container state:

docker inspect web --format '{{.State.OOMKilled}}'

Expected: false normally. If true, the container was killed due to memory limit. Increase the limit or optimize the application.

Check container logs for errors:

docker logs web --tail 50

Also verify that the application responds correctly under load. Use a tool like ab or wrk to generate test traffic. For example, with ab:

ab -n 1000 -c 10 http://localhost/

Observe response times and error rates. For instance, an ab output might show:

Requests per second:    150.23 [#/sec] (mean)
Time per request:       6.657 [ms] (mean)
Failed requests:        0

If performance degrades, adjust limits accordingly. Additionally, you can inspect cgroup metrics directly:

cat /sys/fs/cgroup/memory/docker/<container-id>/memory.stat
cat /sys/fs/cgroup/cpu/docker/<container-id>/cpu.stat

These files show detailed memory and CPU throttling statistics, helping diagnose subtle issues.

Failure Modes and Recovery

Resource limits can cause failures if set too low. Common failure modes include OOM kills, CPU throttling, and application timeouts. Plan for these and know how to recover.

Memory Exhaustion

A container exceeding its memory limit is killed by the kernel OOM killer. Symptoms include abrupt container restarts and OOMKilled status true. Recovery: Increase memory limit in Compose file and restart the service.

docker compose up -d --force-recreate web

To prevent recurrence, monitor memory usage trends and set limits with buffer. For example, if your web container peaks at 400 MB, set limit to 512 MB or higher.

CPU Throttling

If a container uses more CPU than its limit, the kernel throttles it, causing slow response. Check CPU throttling metrics via docker stats or cgroup files. Recovery: Increase CPU limit or optimize code.

To see throttling, run:

docker stats --no-stream --format 'table {{.Name}}\t{{.CPUPerc}}'

And inspect:

cat /sys/fs/cgroup/cpu/docker/<container-id>/cpu.stat

Look for nr_throttled increasing. If so, raise the CPU limit.

Disk Space Exhaustion

Containers writing logs or data can fill the host disk. Monitor disk usage with df -h. Configure log rotation in Compose:

services:
  web:
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"

For database containers, ensure data volumes have sufficient capacity and monitor growth.

Rollback Strategy

Keep previous Compose file versions (e.g., using git tags). To rollback resource changes, revert to the previous file and redeploy.

git checkout tags/v1.0 -- docker-compose.yml
docker compose up -d

After recovery, verify with docker stats and application health checks.

Operations Checklist

Use the following checklist for ongoing capacity management. Assign an owner for each item (e.g., DevOps engineer or team lead) and review frequency.

Additional items specific to your application may be needed. For example, if you use a managed database service, include a check for its storage and connection limits.

Common Pitfalls and How to Avoid Them

Capacity planning with Docker Compose often goes wrong in predictable ways. Here are common pitfalls, why they happen, and how to avoid or recover from them.

Ignoring Host-Level Limits

Why it happens: Administrators focus on container limits but forget that the host itself has finite resources. If the sum of container limits exceeds host capacity, the host may slow down or become unresponsive.

How to avoid: Always calculate total resource allocation. Ensure the sum of limits for all services does not exceed 80% of host resources. Leave the rest for OS and Docker overhead.

Recovery: Reduce limits on some services, or add more host resources (vertical scaling).

Setting Limits Too Low

Why it happens: To be safe, you set conservative limits, but the application needs more during peak load.

How to avoid: Use load testing to determine realistic requirements. Set limits at least 20-30% above observed peak usage.

Recovery: Monitor for OOM kills and throttling. Increase limits progressively and redeploy.

Forgetting Log Rotation

Why it happens: Default Docker logging driver (json-file) does not rotate logs by default, so logs can grow indefinitely and consume disk space.

How to avoid: Configure log rotation in the Compose file as shown earlier.

Recovery: If disk is full, manually truncate logs: truncate -s 0 /var/lib/docker/containers/<container-id>/<container-id>-json.log, then add rotation configuration.

Not Accounting for Database Growth

Why it happens: You plan for initial data size but forget that data accumulates over time.

How to avoid: Monitor database size regularly. Set alerts when disk usage exceeds 70%. Plan for data archival or scaling.

Recovery: Attach larger volume or move to external database service.

Overlooking CPU Throttling

Why it happens: CPU throttling is subtle; the container runs but slower than expected. It may be mistaken for application inefficiency.

How to avoid: Monitor nr_throttled in cgroup stats. Set CPU limits based on actual usage with headroom.

Recovery: Increase CPU limit or optimize application code.

Using V1 Compose Syntax

Why it happens: Some teams still use docker-compose (V1), which does not support the deploy.resources section in the same way.

How to avoid: Migrate to Docker Compose V2 (docker compose). Verify with docker compose version.

Recovery: If stuck with V1, use mem_limit and cpus top-level service properties, but note they are less flexible.

Conclusion

Capacity planning for Docker Compose production deployments is essential for stability and cost efficiency. By inventorying your environment, setting explicit resource limits, verifying with observable metrics, preparing for failure modes, following a repeatable checklist, and avoiding common pitfalls, you can manage capacity proactively.

Next steps: Apply this guide to a small, measurable pilot service, monitor its behavior, and gradually expand to other services. Revisit your capacity plan regularly as your application grows. With these practices, you can avoid common resource pitfalls and ensure your Docker Compose deployment runs smoothly in production.

Capacity planning is an ongoing discipline. Document your decisions, measure actual usage, and adjust limits as needed. With a systematic approach, you can maintain a resilient and efficient deployment.