## Intro Docker Compose is a powerful tool for defining and running multi-container applications. With a single YAML file, you can configure services, networks, and volumes, and start everything with one command. But when environment variables are misconfigured, the results can be confusing: containers fail to start, services connect to the wrong database, or secrets leak into logs. This article focuses on practical, real-world errors developers and operators encounter when using environment variables in Docker Compose. Instead of abstract theory, you will find concrete commands, configuration snippets, and expected outputs. Each section targets a common failure mode, explains why it happens, and shows how to fix it safely. By the end, you will have a repeatable diagnostic workflow that moves from observation to verified resolution without guesswork. We will cover version and environment inventory, safe configuration paths, verification and diagnostics, failure modes and recovery, a comprehensive operations checklist, and common pitfalls. Throughout, the emphasis is on operational safety: observe before changing, limit blast radius, protect secrets, verify results, and document recovery paths. ## Version and Environment Inventory Before changing anything, know exactly what you are running. Docker Compose syntax and behavior have evolved; a variable that works in one version may behave differently in another. Start by recording the exact versions and inspecting the current state of your project. ### Check Docker and Compose versions Run: ```bash docker --version docker compose version ``` Expected output: ``` Docker version 24.0.7, build afdd53b Docker Compose version v2.23.0-desktop.1 ``` Older versions of Compose (the standalone `docker-compose` binary) use a different command syntax and may not support newer features like `env_file` with interpolation or certain health check options. If you are on version 1.x, consider migrating to the Compose v2 plugin (`docker compose`) for better compatibility and performance. ### Inventory the current container environment To see what is actually running, use: ```bash docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}" ``` Example output: ``` NAMES STATUS PORTS web Up 2 minutes 0.0.0.0:8080->80/tcp db Up 2 minutes 5432/tcp ``` For a deeper look at a specific container's environment variables, run: ```bash docker inspect --format '{{range .Config.Env}}{{println .}}{{end}}' ``` This prints the variables exactly as Docker sees them, including any that were substituted by Compose. If you have a running Compose project, use: ```bash docker compose ps docker compose logs -f docker compose exec sh ``` The `exec` command gives you a shell inside the container to inspect variables directly, e.g., `echo $DATABASE_URL`. ### Data persistence: volumes vs. bind mounts A common mistake is losing data because it was written to the container's writable layer instead of a volume. Before changing configuration, confirm where data lives. - A named volume, e.g., `app_data:/var/lib/app`, is managed by Docker and persists across container rebuilds. - A bind mount, e.g., `./data:/var/lib/app`, maps a host directory directly and is useful for development but can suffer from permission issues when the host path does not exist or has the wrong ownership. Run a restart test to verify persistence: ```bash docker compose stop docker compose up -d # Check if data is still there docker compose exec app ls /var/lib/app ``` If the data is missing, the service was writing to the container layer; define a volume in your Compose file and retest. ### Blast radius and recovery Document the intended change in a place where teammates can see it (a ticket, a git branch, a comment in the Compose file). Always make one scoped change at a time and verify before moving on. For example, if you suspect a variable is wrong, change only that variable and then run `docker compose up -d`; do not simultaneously edit ports, networks, and restart policies. ## Safe Configuration Path Many variable-related errors come from placing values in the wrong place or using the wrong syntax. This section walks through the safe way to define and use environment variables in Compose. ### Where variables can be defined Docker Compose supports several sources for environment variables, and they have a specific precedence (from highest to lowest): 1. Shell environment variables (set in the terminal before running `docker compose`) 2. The `environment` section in the Compose file 3. The `env_file` option 4. A `.env` file in the project directory 5. Default values from `docker-compose.yml` (e.g., `${VAR:-default}`) A common error is expecting a value from one source to be visible when another source overrides it. For example, if you set `POSTGRES_PASSWORD` in your shell and also define it in the `environment` section, the shell value wins only if the `environment` section does not also set it. Actually, Compose v2 resolves variables at parse time using the shell environment and the `.env` file; values in `environment` are passed to the container as-is after interpolation. Let's be precise: `environment:` entries can reference `${VAR}` and will be substituted using the shell environment or `.env` file. ### Best practices for safe configuration - **Use a `.env` file** for local development and CI. It keeps variables out of the main Compose file and can be git-ignored if it contains sensitive data. - **Never commit real credentials**. Use placeholders in example files (e.g., `DB_PASSWORD=change-me`) and document how to set them. - **Use `env_file` for large sets of variables** that are shared across services, but be aware that variables in `env_file` override those in `environment` (the opposite of shell variables). - **Prefer explicit mapping in `environment`** for critical settings to make them visible in the Compose file. ### A safe configuration example Suppose you have a web service that needs a database connection string. A fragile setup looks like this: ```yaml services: web: image: myapp:latest environment: DATABASE_URL: postgres://user:secret@db:5432/mydb ``` Hardcoding credentials in YAML is dangerous and prevents reuse. A better approach: In the project root, create a `.env` file: ``` DB_USER=appuser DB_PASSWORD=changeme DB_HOST=db DB_PORT=5432 DB_NAME=mydb ``` In your `docker-compose.yml`, use interpolation: ```yaml services: web: image: myapp:latest environment: DATABASE_URL: postgres://${DB_USER}:${DB_PASSWORD}@${DB_HOST}:${DB_PORT}/${DB_NAME} db: image: postgres:15 environment: POSTGRES_USER: ${DB_USER} POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_DB: ${DB_NAME} volumes: - db_data:/var/lib/postgresql/data volumes: db_data: ``` Run `docker compose config` to preview the fully rendered configuration: ```bash docker compose config ``` This should output the YAML with actual values substituted. Verify that `DATABASE_URL` shows the correct string. If a variable is missing, Compose will either error out or substitute an empty string, which may cause subtle failures. ### Environment variables and secrets For sensitive values like database passwords or API keys, Docker recommends using secrets. In Compose, you can define secrets in the `secrets` section and reference them in the container as files under `/run/secrets/`. For example: ```yaml services: db: image: postgres:15 secrets: - db_password environment: POSTGRES_PASSWORD_FILE: /run/secrets/db_password secrets: db_password: file: ./db_password.txt ``` This avoids putting the password directly in the environment or the Compose file. The Postgres image supports the `_FILE` variant for several variables, which is more secure. ## Verification and Diagnostics After any change, you must verify that the variable has the intended effect inside the running container. This section shows concrete diagnostic commands and how to interpret their output. ### Inspect variables in a running container The most direct check is to run `env` inside the container: ```bash docker compose exec web env | sort ``` Look for the variable you set. For example, you should see `DATABASE_URL=postgres://appuser:changeme@db:5432/mydb`. If you see an empty value or the literal `${DB_USER}`, interpolation failed because the variable was not defined in the shell or `.env` file. ### View logs for configuration errors Containers often fail during startup when required variables are missing. Use: ```bash docker compose logs --tail=50 web ``` Common error messages: - `Error: Database is uninitialized and superuser password is not specified.` (Postgres) - `panic: environment variable DATABASE_URL must be set` (Go apps) - `KeyError: 'SECRET_KEY'` (Python apps) If you see such errors, the container likely exited. Check its status with `docker compose ps`; an exited container shows `Exited (1)` in the status column. ### Using docker inspect for deeper diagnostics `docker inspect` gives you the exact environment array: ```bash docker inspect --format '{{range .Config.Env}}{{println .}}{{end}}' ``` This is useful when you suspect that Compose is not substituting variables as expected. The output is a flat list of `KEY=value` pairs; note that values are not masked, so be careful not to expose secrets in logs or screenshots. ### Health checks and readiness A container may start but fail to connect to a dependency because the dependency is not ready. Use health checks to make your services resilient. Example for a web service: ```yaml services: web: image: myapp:latest depends_on: db: condition: service_healthy healthcheck: test: ["CMD", "curl", "-f", "http://localhost/health"] interval: 30s timeout: 10s retries: 5 ``` Then verify health status: ```bash docker compose ps ``` A healthy service shows `Up (healthy)` in the status column. ### Restart test for persistence As mentioned earlier, restart the container to ensure configuration persists: ```bash docker compose restart ``` Then check that the application still sees the expected files and variables. If a variable is set only in the shell and you restart without it, the value will be lost. Always define variables in the Compose file or `.env` for consistent restarts. ## Failure Modes and Recovery This section describes specific failure modes related to environment variables, how to recognize them, and how to recover. ### Failure 1: Variable not substituted - literal `${VAR}` appears **Symptom:** Application logs show `DATABASE_URL=postgres://${DB_USER}:...` literally, or the container fails with "invalid connection string". **Cause:** The variable `DB_USER` is not defined in the shell or `.env` file, and no default value was provided in the Compose file. Compose leaves the string untouched. **Recovery:** 1. Ensure `.env` file exists and contains the variable. 2. Run `docker compose config` to verify interpolation. If you see `${DB_USER}` in the output, the variable is not set. 3. Export the variable in the shell or add to `.env`, then run `docker compose up -d` again. ### Failure 2: Wrong value due to precedence **Symptom:** A variable has a different value than expected, causing connection failures or wrong behavior. **Cause:** The same variable is defined in multiple places (shell, `.env`, `environment`, `env_file`), and the precedence is misunderstood. **Recovery:** 1. Run `docker compose config` to see the resolved values. 2. Trace the source of the value using the precedence chain. Remove duplicate definitions to simplify. 3. Document the intended source for each variable in a README. ### Failure 3: Secrets exposed in Compose file or environment **Symptom:** Credentials are visible in `docker inspect` or in the Compose file, which is committed to git. **Cause:** Hardcoded secrets. **Recovery:** 1. Move secrets to a separate file (e.g., `.env` or Docker secrets). 2. Reference them via interpolation or `_FILE` variables. 3. Rotate the exposed credentials immediately. 4. Add `.env` to `.gitignore` if it contains secrets, and provide a `sample.env` with placeholders. ### Failure 4: Missing required variable - container exits immediately **Symptom:** `docker compose up` starts, but the service exits with code 1 and logs show a missing variable error. **Cause:** The application expects an environment variable that is not provided. **Recovery:** 1. Check `docker compose logs ` for the error message. 2. Add the variable to the `environment` section or `.env` file. 3. Restart: `docker compose up -d`. 4. Verify with `docker compose exec env`. ### Failure 5: Variable set but service still fails to connect **Symptom:** Application cannot reach a database or other service even though the variable looks correct. **Cause:** Network issues, service not ready, or wrong hostname. **Recovery:** 1. Ensure the service name in the Compose file matches the hostname used in the connection string. Compose service names are DNS resolvable. 2. Add `depends_on` with health checks to enforce startup order. 3. Check network isolation: services on the default network should be able to reach each other by name. If you defined custom networks, ensure both services are on the same network. 4. From inside the container, test connectivity: `docker compose exec web ping db` (if ping is available) or use `nc -zv db 5432`. ## Common Pitfalls and How to Avoid Them Beyond specific failures, several recurring mistakes trip up even experienced users. Understanding these pitfalls saves time and prevents production incidents. ### Pitfall 1: Mixing `.env` and shell variables inconsistently **Why it happens:** Developers set variables in their shell for one session and forget to update `.env` for others. When the next person runs the project, variables are missing. **How to avoid:** Use `.env` as the single source of truth for local development. Document that developers should run `source .env` if needed, but better to rely on Compose's automatic loading of `.env` from the project directory. For CI, define variables in the CI environment. ### Pitfall 2: Using `env_file` incorrectly **Why it happens:** Users think `env_file` is the same as `.env`, but they have different roles. `env_file` injects variables into the container at runtime; `.env` is used by Compose for interpolation in the Compose file itself. **How to avoid:** Use `env_file` only when you need to pass a set of variables to a container from a file. Use `.env` for values that need to be interpolated into the Compose file. Keep them separate to avoid confusion. If you need both, you can specify multiple `env_file` entries. ### Pitfall 3: Forgetting to quote values with special characters **Why it happens:** YAML is sensitive to special characters like `#`, `:`, `{}`, `[]`, `*`, `&`, etc. An unquoted value may be parsed incorrectly. **Example:** ```yaml environment: - PASSWORD=pa$$word # wrong, $$ may be interpreted - PASSWORD: 'pa$$word' # correct ``` **How to avoid:** Always quote values that contain special characters. Use single quotes to prevent any variable expansion in YAML, or double quotes if you need to escape sequences. Better yet, use a `.env` file where quotes are not required. ### Pitfall 4: Relying on host environment variables in production **Why it happens:** Developers test with variables exported in their shell and then deploy to production without setting them, causing failures. **How to avoid:** In production, define all required variables in the orchestrator (e.g., Docker Compose file, Kubernetes secrets, or a config management system). Never depend on shell exports in a production environment. Use a CI/CD pipeline that injects variables from a secret store. ### Pitfall 5: Not validating the Compose file before applying **Why it happens:** Developers edit the YAML and run `docker compose up`, only to see a syntax error or unexpected behavior. **How to avoid:** Always run `docker compose config` after editing the Compose file. It validates the syntax and shows the interpolated result. If there is an error, it will be reported immediately. Additionally, you can use `docker compose config --quiet` to suppress output and just check for errors. ### Pitfall 6: Ignoring the restart policy and environment changes **Why it happens:** Changing environment variables in the Compose file does not automatically recreate containers if they are already running; you must run `docker compose up -d` to apply changes, and even then, Compose may not recreate the container if it thinks nothing changed. Environment changes do trigger recreation, but if you use `docker compose start` instead, variables are not applied. **How to avoid:** After modifying variables, run `docker compose up -d` and check that the container was recreated (look at the container ID or uptime). If not, use `docker compose up --force-recreate -d` to force recreation. ## Operations Checklist Use this checklist as a repeatable procedure when diagnosing environment variable issues in Docker Compose. Each item is a concrete action with a verification command. ### Phase 1: Observe and Inventory - [ ] Run `docker --version` and `docker compose version` and record the output. - [ ] Run `docker compose ps` to see service status. - [ ] For a specific container, run `docker inspect --format '{{range .Config.Env}}{{println .}}{{end}}'` to list environment variables. (Be mindful of secrets; mask them if logging.) - [ ] Check the Compose file and any `.env` files in the project directory. Run `docker compose config` to see the full rendered configuration. ### Phase 2: Identify the Problem - [ ] Compare the expected variable value with the actual value from `docker compose config` or `docker inspect`. - [ ] Check logs for missing variable errors: `docker compose logs --tail=100 `. - [ ] Determine if the variable is defined in multiple places and which source takes precedence. ### Phase 3: Make One Scoped Change - [ ] Modify only the variable in question. Use the safest source: `.env` for interpolation, `environment` for direct container values, or secrets for credentials. - [ ] If using secrets, ensure the secret file exists and is referenced correctly. - [ ] Validate the change with `docker compose config`. ### Phase 4: Apply and Verify - [ ] Apply the change: `docker compose up -d` (or `--force-recreate` if necessary). - [ ] Check service status: `docker compose ps`. - [ ] Verify the variable inside the container: `docker compose exec printenv `. - [ ] Test the application functionality that depends on the variable (e.g., connect to the database). ### Phase 5: Document and Recover - [ ] Record the change in a version-controlled file (e.g., update `.env.example` if applicable). - [ ] Update any runbooks or documentation with the correct variable setting. - [ ] If the change breaks something, roll back: restore the previous value, run `docker compose up -d`, and verify the original behavior. - [ ] Notify the team of the change and any required actions (e.g., they may need to pull the latest `.env`). Each step should be followed in order, and no step should be skipped. This checklist can be adapted to your team's workflow, but the core principle remains: observe, change one thing, verify, document. ## Conclusion Environment variables are a small but critical part of Docker Compose configuration. A single missing or incorrect variable can bring down an entire stack, yet these issues are often the result of simple, avoidable mistakes. By following the diagnostic and recovery procedures outlined here, you can resolve variable-related problems quickly and safely. The key takeaways: - Always start with a version and state inventory: know what is running and where variables come from. - Use a safe configuration path: prefer `.env` for interpolation and secrets for credentials. - Verify every change with `docker compose config`, logs, and `printenv` inside the container. - Understand failure modes: variable not substituted, wrong precedence, exposed secrets, missing variables, and connectivity issues. - Avoid common pitfalls: inconsistent sources, YAML quoting, host dependence, and skipping validation. - Follow the operations checklist to standardize your response. Remember that operational safety is paramount: do not make multiple changes at once, protect sensitive values, and always have a documented recovery path. With practice, you will be able to diagnose and fix environment variable errors in Docker Compose with confidence, reducing downtime and frustration. As a next step, pick one of your existing Compose projects and run through the checklist. Compare the rendered configuration with your expectations, and fix any discrepancies. Then, set up a simple health check and restart test to ensure your services are robust. Finally, review your secret management approach and move credentials out of plain environment variables where possible. By doing this, you will harden your Compose setup and build a habit of verifying before trusting.