## Intro Docker bind mounts let containers read and write files directly on the host filesystem. They are simple to configure and useful for development, but they also introduce host-level dependencies that can fail silently: missing directories, permission drift, disk exhaustion, and accidental deletion. Monitoring bind mounts means observing both the container and the host path it depends on, then turning those observations into actionable metrics, alerts, and runbooks. This guide is written for developers, DevOps consultants, and technical startup teams who already use Docker and need a practical approach to bind mount monitoring. You will learn how to inventory your environment, inspect bind mounts, collect relevant metrics, define alert thresholds, and respond to common failure modes such as permission errors, stale files, and disk pressure. Every section includes concrete commands, expected output, and recovery steps. The goal is operational safety: observe before changing, limit the blast radius, use placeholders instead of secrets, verify the result, and document how to recover if the expected state is not reached. ## Version and Environment Inventory Before you can monitor bind mounts, you need to know exactly what is running. Start with a read-only inventory of your Docker environment and the mounts it uses. **Check Docker version and runtime** ```bash docker version --format '{{.Server.Version}}' ``` Expected output: a version string such as `24.0.7`. Bind mount behavior is stable across recent Docker versions, but permission handling can differ between rootless and rootful setups, so note the security model. ```bash docker info --format '{{.SecurityOptions}}' ``` If you see `name=rootless`, your containers run with user namespace restrictions that affect bind mount permissions. **List running containers and their status** ```bash docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}" ``` Capture this output before making any changes. It tells you which services are up and which ports they expose. **Inspect a container's mounts** ```bash docker inspect --format '{{ json .Mounts }}' ``` The output is a JSON array. For a bind mount it looks like: ```json [ { "Type": "bind", "Source": "/host/data", "Destination": "/app/data", "Mode": "rw", "RW": true, "Propagation": "rprivate" } ] ``` Key fields to monitor: - `Source`: the host path. This must exist, be readable/writable by the container user, and have enough free space. - `Destination`: the path inside the container. - `Mode` / `RW`: whether the mount is read-write or read-only. Read-only mounts prevent accidental writes but may break the application. - `Propagation`: normally `rprivate`, but shared propagation (`rshared`, `rslave`) is needed if the host directory is itself a mount point and changes must propagate. **Identify bind mounts vs named volumes** A named volume is managed by Docker and stored under `/var/lib/docker/volumes/`. A bind mount references an arbitrary host path. You can list volumes and compare: ```bash docker volume ls ``` For Compose projects, `docker compose config` shows the effective mount configuration: ```bash docker compose config --format json ``` Check the `volumes` section for bind mount syntax (`./host/path:/container/path`) versus named volume syntax (`volume_name:/container/path`). **Restart test to validate persistence** A quick way to confirm that data is actually surviving on the host is a restart test. Do this in a staging environment, not production. 1. Stop the container: `docker stop ` 2. Recreate it with the same mount: `docker start ` or `docker compose up -d` 3. Exec into the container and verify the file exists: `docker exec ls -l /app/data/important.txt` If the file is missing after recreation, the application was probably writing to the container filesystem instead of the bind mount. ## Safe Configuration Path Once you know what is running, you can adjust monitoring-related configuration without risking the application. The safe path follows a pattern: observe, capture state, make one scoped change, verify, and roll back if needed. **Read-only observation commands** Use these commands to gather information without changing anything: - `docker ps -a` - see all containers, including stopped ones. - `docker logs --tail 100` - read recent application logs for mount-related errors. - `docker inspect ` - full JSON config, including mounts, environment variables, and health status. - `docker stats --no-stream` - quick CPU, memory, and I/O snapshot. High I/O may indicate bind mount activity. **Check bind mount permissions and ownership** Permission issues are a common cause of bind mount failure. Run this on the host: ```bash ls -ld /host/data stat -c '%U %G %a' /host/data ``` Compare with the user the container runs as. By default, many containers run as root (UID 0), but best practice is to run as a non-root user. If the host directory is owned by `root:root` with mode `755`, a container user with UID 1000 may be unable to write. To see the container's effective user: ```bash docker exec id ``` If you need to change the user, do it in the container config, not by changing the host directory permissions globally. For example, in Docker Compose: ```yaml services: app: image: myapp:latest user: "1000:1000" volumes: - ./data:/app/data ``` Then restart the service and verify write access: ```bash docker compose exec app touch /app/data/test.txt && echo "write ok" ``` **Monitor disk space on the host path** Bind mounts consume host disk space. Check free space with: ```bash df -h /host/data ``` Expected output includes a `Use%` column. Set an alert when usage exceeds 80% for the filesystem containing the bind mount. **Configure read-only bind mounts where possible** If the container only needs to read files, mount read-only to reduce risk: ```yaml volumes: - ./config:/app/config:ro ``` Verify with `docker inspect` that `"RW": false` appears for that mount. This prevents accidental deletion or corruption of host data. ## Verification and Diagnostics To monitor bind mounts effectively, you need continuous metrics and periodic checks, not just one-time inspection. This section shows how to collect relevant metrics and diagnose problems. **Collect bind mount metrics with cadvisor** cadvisor is a container monitoring tool that can export container metrics including filesystem usage. Run it as a privileged container: ```bash docker run -d \ --name cadvisor \ -p 8080:8080 \ -v /:/rootfs:ro \ -v /var/run:/var/run:ro \ -v /sys:/sys:ro \ -v /var/lib/docker/:/var/lib/docker:ro \ -v /dev/disk/:/dev/disk:ro \ gcr.io/cadvisor/cadvisor:v0.47.0 ``` Access the web UI at `http://localhost:8080` or query the API. For example, to see filesystem metrics for a container with bind mounts: ```bash curl http://localhost:8080/api/v1.3/docker/ | jq '.stats[0].filesystem' ``` Expected output includes entries with `device` set to the host device backing the bind mount and `usage` in bytes. Compare usage over time to detect growth. **Monitor host disk I/O with iostat** If the bind mount is on a local disk, `iostat` shows read/write activity: ```bash iostat -x 5 /dev/sda ``` Watch for high `%util` (near 100%) or long `await` times, which indicate disk saturation that can slow down containers reading/writing bind mounts. **Diagnose a specific file write failure** If an application reports "Permission denied" when writing to a bind mount, reproduce the write as the container user: ```bash docker exec -u 1000 sh -c 'echo test > /app/data/test.txt' ``` If this fails, check the host directory permissions and ownership. If it succeeds, the issue may be application-specific (e.g., SELinux or AppArmor policies). Check SELinux status: ```bash getenforce ``` If enforcing, you may need to add the `:z` or `:Z` label to the bind mount in Docker: ```yaml volumes: - ./data:/app/data:z ``` Then verify access again. **Check bind mount propagation** If a bind mount needs to see changes made to a mounted subdirectory on the host, propagation matters. For example, if `/host/data` is itself a mount point and you mount it into a container, the container may not see sub-mounts unless you set propagation to `rshared`. Inspect current propagation: ```bash docker inspect --format '{{ range .Mounts }}{{ .Source }} {{ .Propagation }}{{ end }}' ``` If it is `rprivate` and you need shared propagation, recreate the container with `--mount type=bind,source=/host/data,target=/app/data,bind-propagation=rshared`. Test by creating a sub-mount on the host and checking if it appears in the container. ## Failure Modes and Recovery Bind mounts fail for predictable reasons: missing host paths, permission changes, disk exhaustion, and accidental deletion. This section describes each failure mode, how to detect it, and how to recover. **Failure: host directory missing or renamed** If the directory specified as the bind mount source does not exist when the container starts, Docker creates it as an empty directory owned by root. This can lead to confusing behavior: the container starts, but the application sees no data. Detection: - Check the host directory: `ls -ld /host/data` - Inspect container mounts: `docker inspect ` shows the source path. - Look for application errors like "directory not found" or "empty data set". Recovery: 1. Stop the container. 2. Restore or recreate the correct directory on the host with proper ownership and permissions. 3. Start the container and verify the application can read its data: `docker logs --tail 50` should show successful startup. Prevention: - Use a configuration management tool to ensure the host directory exists before starting the container. - In Compose, add a healthcheck that tests file access to the mount. **Failure: permission denied on write** Containers often run as a non-root user (UID 1000 or similar), but the host directory may be owned by another user or by root with restrictive permissions. Detection: - Application logs show `PermissionError: [Errno 13] Permission denied: '/app/data/file.txt'`. - Host directory permissions: `ls -ld /host/data` shows `drwxr-xr-x root root`, which does not allow writes for UID 1000. Recovery: 1. Change ownership of the host directory to the container user's UID/GID, or add an ACL if needed. - `sudo chown -R 1000:1000 /host/data` (adjust UID/GID to match container user). 2. Restart the container and test write access: - `docker exec -u 1000 touch /app/data/test.txt` 3. Remove the test file after verification. Prevention: - Set a consistent user in the Dockerfile or Compose file. - Document the required host permissions in the project README. **Failure: disk space exhausted** Bind mounts live on host filesystems, which may fill up due to logs, large uploads, or unconstrained data growth. Detection: - `df -h /host/data` shows `Use%` at 100%. - System logs may show `No space left on device`. - Applications may fail to write or crash. Recovery: 1. Identify large files on the bind mount: `du -sh /host/data/* | sort -rh | head -n 10` 2. Delete or archive unnecessary files after confirming they are not needed. 3. Expand the filesystem or move the bind mount to a larger disk if needed. 4. Restart affected containers and verify writes succeed. Prevention: - Set up disk usage alerts at 80% capacity. - Use log rotation inside containers so logs do not consume host space. - Consider moving large data to a separate dedicated mount or object storage. **Failure: accidental deletion of host files** Because bind mounts are just host directories, a user or another process can delete files outside Docker, and the container will see them disappear immediately. Detection: - Application logs show "file not found" errors. - The host directory has missing files or directories. Recovery: - Restore from backups. If no backup, you may be able to recover deleted files using filesystem tools only if the filesystem supports it (e.g., extundelete on ext4, but success is not guaranteed). - To prevent, use read-only bind mounts where possible, and restrict host access to the directory. **Failure: SELinux or AppArmor blocking access** On systems with mandatory access control enabled, containers may be denied access to bind mounts even when permissions appear correct. Detection: - Audit logs: `ausearch -m avc -ts recent` on RHEL/CentOS shows `denied` entries for the container trying to access the mount. - The container may fail to start or may show permission errors. Recovery: - For SELinux, add the `:z` or `:Z` label to the bind mount in the run command or Compose file: - `docker run -v /host/data:/app/data:z ...` - Or `-v /host/data:/app/data:Z` if the mount is exclusive to this container. - For AppArmor, check the profile applied and adjust if necessary, or use an unconfined profile for testing only. - Restart the container and verify access. **Failure: inode exhaustion** Even if disk space is available, a filesystem can run out of inodes if the application creates many small files. Detection: - `df -i /host/data` shows `IFree` near 0. - Creating a new file fails with "No space left on device" even though `df -h` shows space available. Recovery: - Find directories with many files: `find /host/data -type f | wc -l`, then inspect subdirectories. - Delete unneeded small files or move the directory to a filesystem with more inodes. - Prevent by setting limits on file creation in the application. ## Operations Checklist Use this checklist to keep bind mounts healthy. Run it manually or automate it with cron or CI. **Daily checks** - [ ] Verify all containers with bind mounts are running: `docker ps -a --filter "status=exited"` should not list expected running containers. - [ ] Check disk space on bind mount filesystems: `df -h /host/data` for each host path. - [ ] Check application logs for mount-related errors: `docker logs --since 24h | grep -i "permission\|no space\|not found"` **Weekly checks** - [ ] Confirm bind mount sources exist and permissions are correct: `ls -ld /host/data` and `stat -c '%U %G %a' /host/data`. - [ ] Check for orphaned bind mounts (mounts whose directories no longer exist): compare `docker inspect` mount list with actual host directories. - [ ] Review disk I/O metrics from cadvisor or iostat for unusual activity. **Monthly proactive tasks** - [ ] Test restoration from backup of critical bind mount data. - [ ] Run a read-only test: mount a copy of the data read-only and confirm the application still works. - [ ] Review bind mount propagation needs: if host directories are themselves mounts, ensure container propagation is set correctly. **Before any container change** - [ ] Record current state: `docker ps -a`, `docker inspect --format '{{ json .Mounts }}' `, `df -h`, `du -sh /host/data`. - [ ] Make one scoped change at a time. - [ ] Verify the change with a specific command and expected output. - [ ] Have a rollback plan: keep the old container config or Compose file version. **Incident response quick reference** - [ ] Container cannot write: check permissions, SELinux/AppArmor, disk space, read-only flag. - [ ] Data missing: check if host directory was renamed or deleted, restore from backup. - [ ] Performance degradation: check disk I/O, number of small files, and inode usage. ## Conclusion Monitoring Docker bind mounts is about understanding the host-container relationship and turning it into routine checks and alerts. By inventorying your environment, collecting relevant metrics, and preparing for common failures, you reduce downtime and data loss. Start with one low-risk verification from this guide: run `docker inspect` on a container using a bind mount, check the host directory permissions, and run a restart test to confirm persistence. Document the results and set up a basic disk space alert. Then expand to include cadvisor metrics and automated permission checks. Remember: bind mounts are powerful but fragile. Observe before changing, limit changes to one scoped item, always verify the outcome, and keep backups of critical host data.