## Intro

Docker containers share the host kernel and, by default, can consume as much CPU and memory as the host allows. Without explicit resource limits, a single container can starve other containers or even the host operating system, leading to performance degradation or outages. Configuring resource limits is essential for predictable performance and stability, but many teams make mistakes that either fail to enforce limits or over-restrict containers, causing crashes or poor behavior.

This guide walks through common Docker resource limit configuration mistakes, shows how to set limits correctly with practical examples, and provides validation, rollback, and troubleshooting workflows. By following these steps, you can avoid resource contention issues and ensure your containers run reliably in production.

## Version and Environment Inventory

Before applying resource limits, document your Docker version, host configuration, and container topology. This inventory helps reproduce issues and understand the impact of changes.

### Check Docker Version and Info

Run the following commands to verify Docker version and server details:

```bash
docker version
```

Expected output includes client and server versions. For example:

```
Client: Docker Engine - Community
 Version:           24.0.7
 API version:       1.43
 Go version:        go1.20.10
 Git commit:        afdd53b
 Built:             Thu Oct 26 09:08:17 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.7
```

Confirm your Docker version supports the resource limit flags (`--cpus`, `--memory`). These flags are available since Docker 1.13 and later. If you are on an older version, upgrade first.

```bash
docker info
```

Check the Cgroup Driver and Cgroup Version in the output. Resource limits rely on cgroups. Example snippet:

```
Cgroup Driver: systemd
Cgroup Version: 2
```

### Host Resource Inventory

List the host's CPU and memory to understand available capacity:

```bash
lscpu | grep -E '^CPU\(s\):|^Model name'
free -h
```

Example output:

```
CPU(s):              8
Model name:          Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
              total        used        free      shared  buff/cache   available
Mem:           31Gi        2.1Gi        27Gi       123Mi       1.6Gi        28Gi
Swap:          2.0Gi          0B       2.0Gi
```

### Container Topology

List running containers and their current resource usage:

```bash
docker ps --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"
docker stats --no-stream
```

`docker stats` shows CPU and memory usage per container. This baseline helps decide appropriate limits.

### Prerequisites

- Docker Engine with cgroups v1 or v2 enabled.
- Root or sudo access to the Docker host for applying limits.
- A test environment to validate changes before production.

## Safe Configuration Path

Setting resource limits incorrectly can hurt performance or cause containers to be killed. Follow a scoped approach: start with a single container in a test environment, measure its behavior, then gradually adjust.

### Common Mistake 1: Not Setting Any Limits

Without limits, a container can consume all CPU and memory. To set limits for a new container:

```bash
docker run -d --name web --cpus="1.5" --memory="512m" nginx
```

This command restricts the container to 1.5 CPU cores and 512 MB of memory.

### Common Mistake 2: Misunderstanding CPU Limits

Using `--cpus` sets a quota on CPU time. For example, `--cpus="1.5"` limits the container to 150% of one CPU core. If you need to pin to specific cores, use `--cpuset-cpus`:

```bash
docker run -d --name cpu-pinned --cpuset-cpus="0-1" nginx
```

However, using `--cpuset-cpus` without `--cpus` still allows the container to use 100% of the pinned cores. Combine both for strict control:

```bash
docker run -d --name cpu-limited --cpus="1.0" --cpuset-cpus="0-1" nginx
```

This limits to one CPU's worth of time across cores 0 and 1.

### Common Mistake 3: Setting Memory Limit Too Low

If a container exceeds its memory limit, the kernel OOM (out-of-memory) killer terminates the process. Set memory limits based on actual usage with headroom. Use `--memory-reservation` as a soft limit that Docker uses for scheduling but does not hard-kill:

```bash
docker run -d --name app --memory="1g" --memory-reservation="750m" myapp
```

This sets a hard limit of 1 GB and a soft reservation of 750 MB.

### Common Mistake 4: Forgetting Swap Limits

By default, a container can use as much swap as the host has available. To control swap, set `--memory-swap` in conjunction with `--memory`. For example, to allow no swap:

```bash
docker run -d --name no-swap --memory="512m" --memory-swap="512m" myapp
```

Here, `--memory-swap` equals `--memory`, so no swap is allowed. To allow 1 GB total (memory + swap), set `--memory-swap="1g"`:

```bash
docker run -d --name with-swap --memory="512m" --memory-swap="1g" myapp
```

### Common Mistake 5: Setting Limits Only at Container Start

Limits can also be set in Docker Compose or via update. For existing containers, you can update limits with:

```bash
docker update --cpus="2.0" --memory="1g" web
```

Note: You cannot update `--memory-swap` on a running container; you must recreate it.

### Docker Compose Example

In `docker-compose.yml`, specify limits under `deploy.resources.limits` (for Swarm) or `mem_limit` and `cpus` (for Compose v2/v3). Use the following for Compose v3 with Swarm-compatible syntax:

```yaml
version: '3.8'
services:
  web:
    image: nginx
    deploy:
      resources:
        limits:
          cpus: '1.5'
          memory: 512M
        reservations:
          cpus: '0.5'
          memory: 256M
```

Deploy with `docker stack deploy -c docker-compose.yml mystack` if using Swarm, or use `docker compose up` (Compose v2) which also respects these limits.

### Step-by-Step Safe Implementation

- **Measure baseline**: Run the container without limits and monitor `docker stats` for typical and peak usage.
- **Apply conservative limits**: Start with limits 20-30% above observed peak.
- **Load test**: Simulate realistic traffic and monitor for throttling or OOM kills.
- **Adjust gradually**: Increase or decrease limits based on observed behavior.
- **Document and version control**: Store configuration in Docker Compose or Kubernetes manifests.

## Verification and Diagnostics

After setting limits, verify they are enforced and diagnose any issues.

### Inspect Container Configuration

Use `docker inspect` to see the applied limits:

```bash
docker inspect web --format '{{.HostConfig.NanoCpus}} {{.HostConfig.Memory}}'
```

For example, output:

```
1500000000 536870912
```

`NanoCpus` is in billionths of a CPU (1.5 x 1e9 = 1500000000), and `Memory` is in bytes (512 MB = 536870912).

### Check cgroup Limits Directly

Access the container's cgroup files on the host. For cgroup v2, the path is `/sys/fs/cgroup/system.slice/docker-<container-id>.scope/`. Find the container ID with `docker ps --no-trunc`. Then check:

```bash
cat /sys/fs/cgroup/system.slice/docker-<container-id>.scope/cpu.max
```

Example output for a limit of 1.5 CPUs:

```
150000 100000
```

This means the container can use 150000 microseconds of CPU time per 100000 microseconds period.

For memory:

```bash
cat /sys/fs/cgroup/system.slice/docker-<container-id>.scope/memory.max
```

Output:

```
536870912
```

### Monitor Runtime Statistics

Use `docker stats` to observe live resource usage:

```bash
docker stats web
```

Output includes CPU % and MEM USAGE / LIMIT. Check that MEM USAGE stays below the limit.

### Test CPU Throttling

To see if CPU is being throttled, check the `nr_throttled` counter in the cgroup:

```bash
cat /sys/fs/cgroup/system.slice/docker-<container-id>.scope/cpu.stat | grep nr_throttled
```

If the value is increasing, the container is hitting its CPU limit and being throttled. Consider raising the limit.

### Test Memory OOM

Deliberately exceed the memory limit to see OOM behavior. Run a container with a small memory limit and a command that allocates memory:

```bash
docker run --rm --memory="50m" alpine sh -c 'tail /dev/zero'
```

This command reads from `/dev/zero` indefinitely, consuming memory. After a few seconds, the kernel OOM killer will terminate the process. Check the container exit code:

```bash
docker run --rm --memory="50m" alpine sh -c 'tail /dev/zero'; echo $?
```

Expected exit code is 137 (128 + 9, SIGKILL), indicating OOM kill.

## Failure Modes and Recovery

Resource limits can cause failures if misconfigured. Understand common failure modes and how to recover.

### Failure Mode 1: Container OOM Killed

**Symptom**: Container stops unexpectedly, exit code 137, logs show OOM kill.

**Diagnosis**: Check `docker inspect` for `State.OOMKilled`:

```bash
docker inspect web --format '{{.State.OOMKilled}}'
```

If `true`, the container exceeded its memory limit.

**Recovery**: Increase memory limit or reduce memory usage. Update the limit:

```bash
docker update --memory="1g" web
```

If the container crashed, restart it with corrected settings.

### Failure Mode 2: CPU Starvation

**Symptom**: Application is slow, high latency, but CPU usage on host is not maxed. `nr_throttled` increases.

**Diagnosis**: Check cgroup CPU throttling as shown earlier.

**Recovery**: Increase CPU limit:

```bash
docker update --cpus="2.0" web
```

### Failure Mode 3: Incorrect Swap Configuration

**Symptom**: Container uses excessive swap, causing disk I/O and slowdown.

**Diagnosis**: Check swap usage in `docker stats` (if available) or in cgroup `memory.swap.current`.

**Recovery**: Recreate the container with appropriate `--memory-swap`:

```bash
docker stop web && docker rm web
docker run -d --name web --memory="512m" --memory-swap="512m" nginx
```

### Failure Mode 4: Limit Not Applied

**Symptom**: Container uses more resources than expected, `docker inspect` shows no limits.

**Cause**: Limits specified in a compose file but not applied due to using `docker-compose up` with unsupported fields, or forgetting to deploy the stack.

**Diagnosis**: Inspect the running container:

```bash
docker inspect web --format '{{.HostConfig.NanoCpus}} {{.HostConfig.Memory}}'
```

If values are `0 0`, no limits are set.

**Recovery**: Apply limits explicitly via `docker update` or recreate with correct flags.

### Rollback and Recovery Best Practices

- Keep configuration in version control.
- Use `docker update` for quick limit changes without recreating the container.
- For swap changes, recreate the container with a script or compose file.
- Test rollback by restoring previous limits and verifying with `docker stats`.

## Common Pitfalls and How to Avoid Them

Besides the mistakes already described, here are additional pitfalls that often trip up teams:

### Pitfall 1: Ignoring CPU Throttling Metrics

Many teams set CPU limits but never check whether the container is being throttled. Throttling can cause latency spikes even if CPU usage appears below the limit. **Why it happens**: The container hits the CPU quota within a scheduling period and is forced to wait. **How to avoid**: Regularly monitor `nr_throttled` in cgroup `cpu.stat`. If it increases under normal load, raise the CPU limit or adjust the period (advanced).

### Pitfall 2: Not Setting Memory Reservation

A memory hard limit alone can lead to aggressive swapping or OOM kills under memory pressure. **Why it happens**: The kernel may reclaim pages from the container, causing performance degradation before the hard limit is reached. **How to avoid**: Set a `--memory-reservation` below the hard limit to signal a softer boundary. This helps the kernel make better decisions.

### Pitfall 3: Overcommitting Host Resources

If the sum of container limits exceeds host capacity, the host may become unstable. **Why it happens**: Teams assume limits are reservations, but they are only caps; containers can still compete for resources. **How to avoid**: Use `docker stats` and host monitoring to ensure total limits do not exceed 80-90% of host capacity. Leave headroom for the host OS and system daemons.

### Pitfall 4: Forgetting to Update Compose Files

After manually adjusting limits with `docker update`, teams forget to update the source-of-truth compose file. **Why it happens**: Quick fixes in production lead to configuration drift. **How to avoid**: Always update the compose file or manifest first, then apply changes via a deployment pipeline. Use version control to track changes.

## Operations Checklist

Use this checklist for ongoing management of Docker resource limits. Assign a single owner (e.g., DevOps Engineer) for each recurring task and review frequency.

| Task | Command or Action | Owner | Frequency |
|------|-------------------|-------|-----------|
| Review container resource usage | `docker stats --no-stream` | SRE on call | Weekly |
| Check for OOM kills | `docker inspect --format '{{.State.OOMKilled}}' <container>` | Application owner | After incidents |
| Verify limits in configuration | `docker inspect --format '{{.HostConfig.NanoCpus}} {{.HostConfig.Memory}}' <container>` | DevOps lead | Monthly |
| Audit throttling | Check `cpu.stat` `nr_throttled` per container | Performance engineer | Monthly |
| Update limits based on trends | Adjust `--cpus` and `--memory` as needed | Service owner | Quarterly |
| Test OOM behavior in staging | Run memory stress test in non-production | QA lead | Before major releases |
| Document limits in CMDB | Maintain a record of resource allocations | Configuration manager | Continuously |

### Additional Considerations

- **Set default limits for daemon**: You can configure some default resource limits in `/etc/docker/daemon.json` using `default-ulimits`, but CPU and memory defaults are not directly supported. Use orchestration policies or templates to enforce defaults.
- **Use orchestrator features**: If using Kubernetes, set resource requests and limits in pod specs. For Docker Swarm, use service constraints and resource reservations.
- **Monitor with tools**: Integrate `docker stats` output into monitoring systems like Prometheus via cAdvisor or node exporter for alerting on resource usage and throttling.

## Conclusion

Configuring Docker resource limits is a critical practice for maintaining performance and stability. By understanding common mistakes such as neglecting limits, misunderstanding CPU semantics, setting memory too low, and ignoring swap, you can avoid production issues. The safe configuration path involves measuring baseline usage, applying conservative limits, load testing, and gradually adjusting. Verification through `docker inspect`, cgroup files, and `docker stats` ensures limits are effective. When failures occur, quick recovery is possible with `docker update` or container recreation. Finally, an operations checklist with clear ownership and review frequencies helps keep resource management consistent over time.

Start with one container, apply the examples from this guide, and measure the impact. Then expand to your entire fleet with confidence.