Docker Security Production Operations: A Practical Checklist with Examples

Intro

Running Docker in production requires more than building and shipping images. Security is an ongoing operational responsibility that spans configuration, monitoring, maintenance, backups, and upgrades. Without a repeatable process, teams risk exposure to container escapes, privilege escalation, secret leaks, and supply-chain attacks.

This article provides a practical operations checklist for Docker security in production. Each section includes concrete commands, configuration snippets, expected outputs, and verification steps. The goal is to give developers, DevOps consultants, and technical startup teams a reference they can adapt to their own environments.

We cover:

• Inventorying Docker versions and environment prerequisites
• Applying a safe configuration path with scoped hardening
• Verifying the security posture with observable checks
• Handling failure modes and recovery procedures
• A repeatable operations checklist for ongoing security

By the end, you will have a clear set of actions to harden Docker hosts, enforce runtime policies, detect anomalies, and recover from incidents.

Version and Environment Inventory

Before making any security changes, document the current Docker environment. This includes Docker Engine and CLI versions, container runtime, host operating system, storage and network drivers, and current security-related settings.

Check Docker Versions

Run the following on each Docker host:

docker version

Example output (truncated):

Client: Docker Engine - Community
  Version:           24.0.5
  API version:       1.43
  Go version:        go1.20.4
  Git commit:        ced0996
  Built:             Fri Jul 21 19:01:22 2023
  OS/Arch:           linux/amd64
  Context:           default

Server: Docker Engine - Community
  Engine:
    Version:          24.0.5
    API version:      1.43 (minimum version 1.12)
    Go version:       go1.20.4
    Git commit:       a61e2b4
    Built:            Fri Jul 21 19:01:22 2023
    OS/Arch:          linux/amd64
    Experimental:     false
  containerd:
    Version:          1.6.21
    GitCommit:        3dce8eb055cbb6878b1e53c1b34f8f1b3e3f0c1a
  runc:
    Version:          1.1.7
    GitCommit:        v1.1.7-0-g860f061
  docker-init:
    Version:          0.19.0
    GitCommit:        de40ad0

Record the Engine version and the components (containerd, runc). Security patches often target specific versions, so keeping this inventory up to date is critical.

Inspect Current Security Configuration

Use docker info to see general security settings:

docker info --format '{{json .SecurityOptions}}'

Example output:

["name=seccomp,profile=builtin","name=cgroupns"]

If you see name=userns in the output, user namespace remapping is enabled. If not, it may be a candidate for hardening (discussed later).

Also check the default seccomp profile and AppArmor/SELinux status:

docker info | grep -E 'Security Options|Kernel Version|Operating System'

Identify Privileged Containers

List currently running containers and check for privileged mode:

docker ps --quiet | xargs docker inspect --format '{{.Name}}: Privileged={{.HostConfig.Privileged}}'

Example output:

/mysql: Privileged=false
/web: Privileged=true
/cache: Privileged=false

The /web container is running privileged, which is risky. You will address this in the safe configuration path.

Prerequisites

Ensure you have:

• SSH access to Docker hosts with sudo (or equivalent)
• A test or staging environment to validate changes before production
• Backup of Docker daemon configuration (/etc/docker/daemon.json)
• Backup of any systemd unit files or init scripts for Docker
• Ability to restart the Docker daemon (requires scheduling maintenance window)
• Log aggregation and monitoring tools (e.g., ELK, Prometheus, Datadog)

Document the inventory in a spreadsheet or configuration management database. This baseline will be used for verification later.

Safe Configuration Path

Apply security hardening in a controlled, incremental manner. Start with the lowest-risk, highest-impact changes: daemon configuration, container runtime settings, and network policies. Always test on a non-production host first.

1. Harden the Docker Daemon

The Docker daemon runs with root privileges and controls all containers. A misconfigured daemon can expose the host. Edit /etc/docker/daemon.json (create if missing) and add or modify the following:

{
  "icc": false,
  "log-level": "info",
  "userland-proxy": false,
  "no-new-privileges": true,
  "live-restore": true,
  "iptables": true,
  "ip6tables": true,
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  },
  "default-ulimits": {
    "nofile": {
      "Name": "nofile",
      "Hard": 64000,
      "Soft": 64000
    }
  }
}

Key settings:

• "icc": false: Disables inter-container communication by default. Containers must be explicitly linked or on the same user-defined network to communicate.
• "no-new-privileges": true: Prevents containers from gaining additional privileges via setuid/setgid binaries or other mechanisms.
• "live-restore": true: Keeps containers running when the Docker daemon is stopped, reducing downtime during upgrades.

After editing, restart the Docker daemon:

sudo systemctl restart docker

Verify the settings:

docker info --format '{{json .SecurityOptions}}'

Expected output should now include no-new-privileges. Actually, no-new-privileges is a per-container flag; the daemon setting is not shown there. Instead, check docker info for the daemon-level settings:

sudo docker info | grep -i 'Default Runtime\|Cgroup Driver\|Security Options'

But to verify icc: false, run:

docker network inspect bridge | grep -i 'com.docker.network.bridge.enable_icc'

Expected output:

"com.docker.network.bridge.enable_icc": "false"

2. Enable User Namespace Remapping

User namespaces map the container's root user to a non-root user on the host, limiting the impact of container escapes. This is a powerful security control but can break some workloads (e.g., those requiring specific file permissions). Test thoroughly.

To enable, add to /etc/docker/daemon.json:

{
  "userns-remap": "default"
}

Then restart Docker:

sudo systemctl restart docker

Check that the remapped user exists:

id dockremap

Example output:

uid=140000(dockremap) gid=140000(dockremap) groups=140000(dockremap)

Note: If you have existing containers or volumes, they may have permission issues after enabling. Plan to recreate containers and adjust volume ownership. Refer to Docker's documentation for migration steps.

3. Restrict Container Capabilities

By default, Docker containers get a subset of Linux capabilities. You can further reduce privileges by dropping all capabilities and adding back only what is needed. Use the --cap-drop and --cap-add flags.

Example: Run an Nginx container with minimal capabilities:

docker run -d --name web \
  --cap-drop ALL \
  --cap-add NET_BIND_SERVICE \
  --security-opt no-new-privileges \
  nginx:alpine

Here we drop all capabilities, then add only NET_BIND_SERVICE (to bind to port 80). We also set no-new-privileges to prevent privilege escalation.

Verify the capabilities of a running container:

docker inspect --format '{{.HostConfig.CapAdd}} {{.HostConfig.CapDrop}}' web

Expected output:

[NET_BIND_SERVICE] [ALL]

4. Apply Resource Limits

Unlimited resource usage can lead to denial of service. Set memory and CPU limits on containers:

docker run -d --name api \
  --memory 512m \
  --cpus 1.5 \
  --pids-limit 100 \
  myapp:1.0

These flags ensure the container uses at most 512 MB RAM, 1.5 CPU cores, and 100 processes/threads. --pids-limit prevents fork bombs.

Check limits:

docker inspect --format '{{.HostConfig.Memory}} {{.HostConfig.NanoCpus}} {{.HostConfig.PidsLimit}}' api

Expected output:

536870912 1500000000 100

5. Use Read-Only Root Filesystem

Many applications can run with a read-only root filesystem, preventing attackers from writing malicious files. Use --read-only and mount writable volumes for specific paths (e.g., /tmp, /var/run).

docker run -d --name web \
  --read-only \
  --tmpfs /tmp \
  --tmpfs /var/run \
  nginx:alpine

Verify:

docker inspect --format '{{.HostConfig.ReadonlyRootfs}}' web

Expected output:

true

6. Secure the Docker Socket

The Docker socket (/var/run/docker.sock) grants full control over the daemon. Never expose it to containers unless absolutely necessary. If a container needs to manage Docker (e.g., Portainer), use a proxy like docker-socket-proxy with restricted permissions.

Check if any container mounts the socket:

docker ps --quiet | xargs docker inspect --format '{{.Name}}: {{range .Mounts}}{{if eq .Source "/var/run/docker.sock"}}YES{{end}}{{end}}'

If any container shows YES, evaluate whether it truly needs the socket and restrict with a proxy.

7. Network Segmentation

Use user-defined bridge networks to isolate containers. Avoid using the default bridge network in production.

Create a network:

docker network create --driver bridge frontend

Run containers attached to this network:

docker run -d --name web --network frontend nginx:alpine

Container communication across different networks is blocked unless explicitly allowed.

8. Configure Logging and Monitoring

Ensure logs are captured and rotated. The daemon configuration above sets JSON file logging with rotation. For production, consider shipping logs to a central system using a logging driver like fluentd or splunk.

Example daemon.json addition:

{
  "log-driver": "fluentd",
  "log-opts": {
    "fluentd-address": "localhost:24224",
    "tag": "docker.{{.Name}}"
  }
}

Then restart Docker. Test by running a container and checking the log destination.

Verification and Diagnostics

After applying changes, verify that the security controls are effective. Use automated tools and manual checks.

1. Run Docker Bench for Security

Docker Bench for Security is a script that checks for common best practices. Run it on the host:

git clone https://github.com/docker/docker-bench-security.git
cd docker-bench-security
sudo ./docker-bench-security.sh

The output is a long report with [WARN], [INFO], and [PASS] items. Example excerpt:

[INFO] 1 - Host Configuration
[WARN] 1.1 - Ensure a separate partition for containers has been created
[PASS] 1.2 - Ensure the container host has been Hardened
[PASS] 1.3 - Ensure Docker is up to date
[INFO] 2 - Docker daemon configuration
[WARN] 2.1 - Ensure network traffic is restricted between containers on the default bridge
[PASS] 2.2 - Ensure the logging level is set to 'info'
...

Address warnings as appropriate for your environment. Note that some warnings may be acceptable due to operational constraints; document exceptions.

2. Scan Images for Vulnerabilities

Use image scanning tools like Trivy, Clair, or Docker Scout. Example with Trivy:

trivy image nginx:alpine

Example output (truncated):

nginx:alpine (alpine 3.18.3)
Total: 0 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 0, CRITICAL: 0)

If vulnerabilities are found, update the base image or apply patches. Integrate scanning into your CI/CD pipeline to catch issues early.

3. Verify Seccomp and AppArmor Profiles

Check that containers are using the default seccomp profile (or a custom one).

docker inspect --format '{{.HostConfig.SecurityOpt}}' web

Expected output includes seccomp=... and possibly apparmor=... if AppArmor is enabled.

If you have a custom seccomp profile, test it by attempting a forbidden syscall inside the container. For example, with a profile that blocks mkdir, running mkdir /tmp/test should fail with Operation not permitted.

4. Test User Namespace Remapping

After enabling userns-remap, run a container and check the ownership of files created in a volume from the host perspective.

docker run -d --name test -v /tmp/testvol:/data alpine sleep 300
docker exec test touch /data/testfile
ls -l /tmp/testvol

Expected output shows testfile owned by dockremap (e.g., 140000) rather than root.

5. Confirm No Privileged Containers

Re-run the privileged container check:

docker ps --quiet | xargs docker inspect --format '{{.Name}}: Privileged={{.HostConfig.Privileged}}'

Output should show Privileged=false for all containers. If any are true, remediate immediately.

6. Check Network Isolation

Test inter-container communication. With icc: false, containers on the default bridge cannot communicate. Create two test containers on the default bridge:

docker run -d --name test1 alpine sleep 300
docker run -d --name test2 alpine sleep 300

Get IP of test1:

docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' test1

Example output: 172.17.0.2

From test2, try to ping test1:

docker exec test2 ping -c 2 172.17.0.2

Expected result: ping fails (100% packet loss) because icc is disabled. If they need to communicate, put them on a user-defined network.

7. Monitor Runtime Anomalies

Use tools like Falco or Sysdig to detect suspicious behavior. Install Falco on the host according to its documentation. Then test by running a container that tries to write to /etc inside the container:

docker run --rm -it alpine sh -c "echo bad > /etc/test"

Falco should generate an alert. Example Falco log line:

12:34:56.789012345: Warning Container with writable /etc directory detected (user=root command=sh container_id=abc123)

Configure alerts to go to your incident response channel.

Failure Modes and Recovery

Security changes can break applications. Plan for failures and know how to roll back quickly.

Common Failure Modes

• Application requires privileges: Some applications legitimately need certain capabilities or even privileged mode (e.g., system daemons, network tools). If you drop capabilities and the app fails, add back only the necessary capabilities.
• User namespace remapping breaks file permissions: Existing volumes may become inaccessible because of UID/GID mapping. Solutions: recreate volumes with correct ownership, or disable userns-remap.
• Read-only root filesystem causes write errors: Apps may try to write to arbitrary locations. Use --tmpfs or volume mounts for required writable paths.
• Network isolation blocks legitimate traffic: Internal services may need communication on the default bridge. Use user-defined networks with explicit connectivity.
• Resource limits cause OOM kills or throttling: Too-low limits can crash containers. Adjust based on observed usage.

Rollback Procedures

For any daemon configuration change, keep a backup of /etc/docker/daemon.json. To rollback:

sudo cp /etc/docker/daemon.json.backup /etc/docker/daemon.json
sudo systemctl restart docker

For container-level changes, simply recreate the container with previous flags.

Document the exact steps to revert each change in your runbook.

Recovery Checks

After rollback, verify that the environment is functioning:

• Run docker ps to ensure containers are up.
• Check application logs for errors.
• Test connectivity.
• Re-run the relevant verification commands from the previous section.

Operations Checklist

Use this checklist for regular security reviews and as part of your operational routine.

Daily Operations

• [ ] Monitor Docker host resource usage (CPU, memory, disk) for anomalies. Owner: On-call DevOps engineer. Check via Prometheus alerts.
• [ ] Check Docker daemon logs (journalctl -u docker) for errors or security events.
• [ ] Review container logs for suspicious activity.
• [ ] Verify that monitoring/alerting (e.g., Falco) is functioning.

Weekly Operations

• [ ] Run Docker Bench for Security and review any new warnings. Owner: Security lead.
• [ ] Scan images for new vulnerabilities (if not integrated into CI/CD).
• [ ] Review user access to Docker hosts and socket permissions. Owner: Infrastructure lead.
• [ ] Check for orphaned containers, images, volumes, and networks; clean up:

docker system prune -f --volumes

Use caution with --volumes as it deletes unused volumes.

Monthly Operations

• [ ] Update Docker Engine and CLI to the latest stable version. Owner: Infrastructure lead, scheduled for first Tuesday of each month.
• [ ] Review and update seccomp/AppArmor profiles if needed.
• [ ] Rotate secrets and credentials used by containers (e.g., registry credentials, API keys).
• [ ] Test backup and restore procedures.
• [ ] Review resource limits and adjust based on performance trends.
• [ ] Evaluate new security features in Docker releases and plan adoption.

Incident Response

In case of a security incident:

• Isolate the affected host or container.
• Capture forensic data: container logs, image metadata, host logs.
• Identify the vulnerability (e.g., exposed port, vulnerable dependency).
• Patch and redeploy.
• Document the incident and update the checklist to prevent recurrence.

Common Pitfalls and How to Avoid Them

1. Ignoring the Docker socket exposure: Many developers mount /var/run/docker.sock into containers for convenience, giving those containers root-equivalent access to the host. Use a proxy like docker-socket-proxy with read-only or restricted API access, and only for containers that truly need it.

1. Running containers as root: By default, Docker containers run as root. If an attacker escapes the container, they get root on the host. Use the USER directive in Dockerfiles to run as a non-root user, and avoid --privileged unless absolutely necessary.

1. Not setting resource limits: Without memory or CPU limits, a single container can exhaust host resources, causing denial of service. Always set --memory and --cpus in production, and use orchestrators like Kubernetes to enforce limits.

1. Using outdated base images: Base images often contain known vulnerabilities. Regularly update images and scan them with tools like Trivy in CI/CD. Pin image versions to avoid unexpected changes.

1. Disabling security features for simplicity: Turning off seccomp, AppArmor, or SELinux to get an app working leaves the host exposed. Instead, create custom profiles that grant only the required syscalls or permissions.

1. Storing secrets in images: Hardcoding passwords or API keys in Dockerfiles or environment variables is a common mistake. Use Docker secrets, Kubernetes secrets, or external secret managers like Vault, and never bake secrets into image layers.

1. Neglecting network segmentation: Relying on the default bridge network allows all containers to communicate. Use user-defined networks with explicit connectivity and consider network policies (e.g., Calico, Cilium) in Kubernetes.

Conclusion

Docker security is not a one-time setup; it requires continuous operational effort. This checklist provides a foundation for hardening your Docker environment, verifying security controls, and recovering from failures. Start with the inventory, apply safe configuration changes incrementally, and establish regular review cycles. Adapt the checklist to fit your specific workloads and risk profile. By following these practices, you can significantly reduce the attack surface of your containerized applications and maintain a robust security posture in production.