[READ-ONLY] Mirror of https://github.com/FoxxMD/blog.
blog.foxxmd.dev
feat: compose env explainer
+416
-6
+416
-6
_posts/2025-04-03-compose-envs-explained.md
+416
-6
_posts/2025-04-03-compose-envs-explained.md
···
3
3
description: >-
4
4
The missing TLDR for how Compose parses ENV variables and passes them to containers
5
5
author: FoxxMD
6
-
date: 2025-04-03 09:00:00 -0400
6
+
date: 2025-04-03 13:00:00 -0400
7
7
categories: [Tips and Tricks]
8
8
tags: [docker, compose, env, environmental variables, interpolation]
9
9
mermaid: true
10
10
pin: false
11
-
published: false
11
+
published: true
12
12
---
13
13
14
+
This guide aims to clear up any ambiguity around how Docker Compose handles environmental variables for its own configuration (`compose.yaml`) and providing ENVs to services.
15
+
16
+
**The key takeaway is that, even though these args/attributes all use the term `env` they are NOT all related to each other: `.env` `--env-file` `env_file:` and `environment:`.**
17
+
18
+
> **Practical Examples** of the scenarios found below can be found and run from [https://github.com/FoxxMD/compose-env-interpolation-example](https://github.com/FoxxMD/compose-env-interpolation-example)
19
+
{: .prompt-tip }
20
+
21
+
## Compose Up Flow
22
+
23
+
Let's start at a high level by describing the different steps Compose takes when `up`ing a stack so that we can see where the disconnect between all of these features takes place.
24
+
14
25
```mermaid
15
26
flowchart TD
16
27
···
19
30
20
31
A[[docker compose up]] --> dockerenv
21
32
subgraph dockerenv [compose.yaml 'black box']
22
-
Par(["Parse <code>.env</code>, <code>--env-file</code> args, and host ENVs"]) --> B("**Interpolates** key-values from previous step into <code>${KEY_LIKE}</code> strings in compose.yaml **file only**")
33
+
Par(["Parse <code>.env</code>, <code>--env-file</code> args, and host ENVs"]) --> B(["**Interpolates** key-values from previous step into <code>${KEY_LIKE}</code> strings in compose.yaml **file only**"])
23
34
end
24
35
B -->|"None of the parsed key-values in the 'black box' above are _automatically_ available below. To re-use them they must either
25
-
1. have been **interpolated** as string literals into compose file or
26
-
2. be re-defined in attributes mentioned below"| C
36
+
1 have been **interpolated** as string literals into compose file or
37
+
2 be re-defined in attributes mentioned below"| C
27
38
C[[Start service creation]] --> servicefoo
28
39
C --> servicebar
29
40
subgraph servicefoo [Service Foo]
···
32
43
subgraph servicebar [Service Bar]
33
44
barenv(["Parses <code>env_file:</code> and <code>environment:</code> attributes in service Bar definition"]) --> barser[[Creates Bar with ENVs from previous step **only**]]
34
45
end
35
-
```
46
+
```
47
+
48
+
Much of the confusion regarding what ENVs are passed to containers stems from a user mistakenly believing that the functionality associated with `.env` and `--env-file` applies to containers, when *by default* it only applies to the literal contents of `compose.yaml`.
49
+
50
+
As noted in the flow above, all of the ENV key-values we parsed from `.env`/`--env-file` for use in [`compose.yaml` variable interpolation](https://docs.docker.com/compose/how-tos/environment-variables/variable-interpolation) are not automatically passed to any created services. They can still be used but to do this we must explicitly define this **again** in each our service definitions.
51
+
52
+
To reiterate **because this is the most important point:** Think of each "parent" box in the flow above as an isolated process that does not know about anything happening in the other boxes. `.env`/`--env-file` apply **only** to the *literal string contents* of `compose.yaml`. Each service cares **only** about its own `env_file:`/`environment:`
53
+
54
+
## ENV Parsing
55
+
56
+
Now, we will cover how each "part" of the above flow actually handles the functionality for parsing/interplating ENVs.
57
+
58
+
### Interpolating into `compose.yaml`
59
+
60
+
When `docker compose up` is run, docker
61
+
62
+
* parses host environmental variables and `.env`/`--env-file` into a list of key-values
63
+
* these key-values are then used these to [interpolate (subtitute)](https://docs.docker.com/compose/how-tos/environment-variables/variable-interpolation/#interpolation-syntax) `${KEY_LIKE}` strings into values **only in the literal string contents of `compose.yaml`**.
64
+
65
+
The order of operations for using these two sources of key-values:
66
+
67
+
* When no `--env-file` arg is given to `compose up` then docker uses a [`.env` file](https://docs.docker.com/compose/how-tos/environment-variables/variable-interpolation/#env-file) located next to `compose.yaml`, if one is present.
68
+
* If an [`--env-file`](https://docs.docker.com/compose/how-tos/environment-variables/variable-interpolation/#substitute-with---env-file) arg is given then it uses that **instead of `.env`**
69
+
* Multiple `--env-file` args can be used to provide multiple files
70
+
* To use both `.env` and `--env-file` simply add `--env-file .env`
71
+
72
+
### Providing ENVs to Containers/Services
73
+
74
+
In **each service** in `compose.yaml` environmental variables need to be **explicitly defined**. None of the key-values we parsed in the above section are automatically passed.
75
+
76
+
To define what ENVs the service will have access to we can use:
77
+
78
+
* `environment:` list with key-values like `MY_KEY: myVal`
79
+
* and/or `env_file:` attribute where you can define a file (or list of files) with key-values that are all loaded into the service
80
+
81
+
Both of these can be used but be aware there is an [order or precedence](https://docs.docker.com/compose/how-tos/environment-variables/envvars-precedence/#how-the-table-works) if the same key is defined more than one place.
82
+
83
+
Additionally, the same files used for `.env`/`--env-file` can be used for `env_file:` **BUT they must be explicitly defined!** To spell this out:
84
+
85
+
If you want
86
+
87
+
```
88
+
MY_VAR=xyz
89
+
```
90
+
{: file='test.env'}
91
+
92
+
To be available to both `compose.yaml` AND the `serviceA` container defined in the stack then you must:
93
+
94
+
`docker compose up --env-file test.env` AND
95
+
96
+
```yaml
97
+
services:
98
+
serviceA:
99
+
env_file: test.env
100
+
# ...
101
+
```
102
+
103
+
## Examples
104
+
105
+
The below examples can all be run using the repository [https://github.com/FoxxMD/compose-env-interpolation-example](https://github.com/FoxxMD/compose-env-interpolation-example) which has all the compose/env files provided.
106
+
107
+
When a container is run from our sample image (used for the examples below and available in the repository) it runs a single command that prints ENVs available in the container:
108
+
109
+
```
110
+
echo "This is a variable: $FOO $BAR $HARD"
111
+
```
112
+
113
+
### `compose.yaml` with `.env` {#compose-with-dot-env}
114
+
115
+
Interpolate key-values from `.env` into a compose file. No environmental variables are passed to the service.
116
+
117
+
<details markdown="1">
118
+
119
+
```yaml
120
+
services:
121
+
test:
122
+
build:
123
+
context: .
124
+
```
125
+
{: file='compose.yaml'}
126
+
127
+
```
128
+
FOO=aSecret
129
+
```
130
+
{: file='.env'}
131
+
132
+
```shell
133
+
docker compose up
134
+
```
135
+
136
+
Output is
137
+
138
+
```
139
+
This is a variable:
140
+
```
141
+
{: file='Container output'}
142
+
143
+
144
+
* Compose file does not define any env attributes for the service so nothing from `.env` is inserted into container
145
+
146
+
</details>
147
+
148
+
### `environment:` in service
149
+
150
+
#### With default `.env`
151
+
152
+
Interpolate key-values from `.env` into a compose file. Environmentable variables are passed to the service using `environment:` where some come from interpolated `.env`
153
+
154
+
<details markdown="1">
155
+
156
+
```yaml
157
+
services:
158
+
test:
159
+
build:
160
+
context: .
161
+
environment:
162
+
FOO: ${FOO}
163
+
BAR: ${BAR}
164
+
HARD: alwaysHere
165
+
```
166
+
{: file='compose-environment.yaml'}
167
+
168
+
```
169
+
FOO=aSecret
170
+
```
171
+
{: file='.env'}
172
+
173
+
```shell
174
+
docker compose -f compose-environment.yaml up
175
+
```
176
+
177
+
Output is
178
+
179
+
```
180
+
This is a variable: aSecret alwaysHere
181
+
```
182
+
{: file='Container output'}
183
+
184
+
185
+
* Uses `.env` for Compose file since no other `--env-file` args
186
+
* Compose file has `environment:` where...
187
+
* `FOO` (container ENV) is set to `${FOO}` from `.env`
188
+
* `HARD` (container ENV) is hardcoded to `alwaysHere` in compose file
189
+
190
+
</details>
191
+
192
+
#### Using explicit `add.env`
193
+
194
+
Interpolate key-values from `add.env` into a compose file. Environmentable variables are passed to the service using `environment:` where some come from interpolated `add.env`
195
+
196
+
<details markdown="1">
197
+
198
+
```yaml
199
+
services:
200
+
test:
201
+
build:
202
+
context: .
203
+
environment:
204
+
FOO: ${FOO}
205
+
BAR: ${BAR}
206
+
HARD: alwaysHere
207
+
```
208
+
{: file='compose-environment.yaml'}
209
+
210
+
```
211
+
BAR=someBar
212
+
```
213
+
{: file='add.env'}
214
+
215
+
216
+
```shell
217
+
docker compose -f compose-environment.yaml --env-file add.env up
218
+
```
219
+
220
+
Output is
221
+
222
+
```
223
+
This is a variable: someBar alwaysHere
224
+
```
225
+
{: file='Container output'}
226
+
227
+
* Uses `add.env` for Compose file because of `--env-file` args
228
+
* Does NOT use `.env` because `--env-file` is given
229
+
* Compose file has `environment:` where...
230
+
* `BAR` (container ENV) is set to `${BAR}` from `add.env`
231
+
* `HARD` (container ENV) is hardcoded to `alwaysHere` in compose file
232
+
233
+
</details>
234
+
235
+
#### Using multiple explicit env files
236
+
237
+
Interpolate key-values from `add.env` into a compose file. Environmentable variables are passed to the service using `environment:` where some come from interpolated `add.env`
238
+
239
+
<details markdown="1">
240
+
241
+
```yaml
242
+
services:
243
+
test:
244
+
build:
245
+
context: .
246
+
environment:
247
+
FOO: ${FOO}
248
+
BAR: ${BAR}
249
+
HARD: alwaysHere
250
+
```
251
+
{: file='compose-environment.yaml'}
252
+
253
+
```
254
+
FOO=aSecret
255
+
```
256
+
{: file='.env'}
257
+
```
258
+
BAR=someBar
259
+
```
260
+
{: file='add.env'}
261
+
262
+
```shell
263
+
docker compose -f compose-environment.yaml --env-file add.env --env-file .env up
264
+
```
265
+
266
+
Output is
267
+
268
+
```
269
+
This is a variable: aSecret someBar alwaysHere
270
+
```
271
+
{: file='Container output'}
272
+
273
+
* Uses `add.env` for Compose file because of `--env-file` arg
274
+
* Uses `.env` for Compose file because of `--env-file` arg
275
+
* Compose file has `environment:` where...
276
+
* `FOO` (container ENV) is set to `${FOO}` from `.env`
277
+
* `BAR` (container ENV) is set to `${BAR}` from `add.env`
278
+
* `HARD` (container ENV) is hardcoded to `alwaysHere` in compose file
279
+
280
+
</details>
281
+
282
+
### Using `env_file:` in service {#ex-env-file-in-service}
283
+
284
+
Uses [`env_file:`](https://docs.docker.com/compose/how-tos/environment-variables/set-environment-variables/#use-the-env_file-attribute) to load all key-values found in the given file into the service's container.
285
+
286
+
> Compare this to the [`compose.yaml` with `.env` example](#compose-with-dot-env)
287
+
{: .prompt-tip }
288
+
289
+
<details markdown="1">
290
+
291
+
```yaml
292
+
services:
293
+
test:
294
+
build:
295
+
context: .
296
+
env_file: .env
297
+
```
298
+
{: file='compose-envfile.yaml'}
299
+
300
+
```
301
+
FOO=aSecret
302
+
```
303
+
{: file='.env'}
304
+
305
+
```shell
306
+
docker compose -f compose-envfile.yaml up
307
+
```
308
+
309
+
Output is
310
+
311
+
```
312
+
This is a variable: aSecret
313
+
```
314
+
{: file='Container output'}
315
+
316
+
* Uses `.env` for Compose file since no other `--env-file` args
317
+
* Compose file has `env_file:` which inserts everything found in the attribute value (`.env`) into the container
318
+
319
+
</details>
320
+
321
+
### Using `.env` for general variable interpolation
322
+
323
+
This guide and examples have so far focused on using `.env` for interpolating environmental variables into containers but [interpolation can be used *anywhere*](https://docs.docker.com/compose/how-tos/environment-variables/variable-interpolation) in `compose.yaml`.
324
+
325
+
This example interpolates `.env` key-values into `${VERSION}` to define what tag to use for a theoretical image.
326
+
327
+
<details markdown="1">
328
+
329
+
```yaml
330
+
services:
331
+
test:
332
+
image: myFakeImage:${VERSION}
333
+
```
334
+
{: file='compose.yaml'}
335
+
336
+
```
337
+
VERSION=v1.2.3
338
+
```
339
+
{: file='.env'}
340
+
341
+
```shell
342
+
docker compose up
343
+
```
344
+
345
+
The **interpolated** `compose.yaml` contents
346
+
347
+
```yaml
348
+
services:
349
+
test:
350
+
image: myFakeImage:v1.2.3
351
+
```
352
+
{: file='compose.yaml'}
353
+
354
+
* Uses `.env` for Compose file since no other `--env-file` args
355
+
* `${VERSION}` is substituted for `v1.2.3` found in `.env`
356
+
* `VERSION` is NOT passed to the service `test` since no env attributes are defined
357
+
358
+
</details>
359
+
360
+
## Komodo and ENVs
361
+
362
+
> Not familiar with Komodo? Check out my [Introduction to Komodo](../migrating-to-komodo) and [FAQ](../komodo-tips-tricks).
363
+
{: .prompt-tip }
364
+
365
+
[Komodo](https://komo.do/) adds one more layer to this process that occurs **before `docker compose up`**.
366
+
367
+
Komodo takes the contents of the **Environment** section in your Komodo Stack and:
368
+
369
+
* Interpolates any [Variables](https://komo.do/docs/variables) into the contents like `[[MY_VAR]]`
370
+
* The final content is written to the file specified by **Env File Path** in your Stack (defaults to `.env`)
371
+
* The file specified by Env File Path is the **ONLY** file that Variable interpolation occurs in
372
+
373
+
After this is done it then runs `docker compose up --env-file .env` and includes any additional files you specified in the **Additional Env Files** section of your stack.
374
+
375
+
<details markdown="1">
376
+
377
+
<summary>Variables Example</summary>
378
+
379
+
Assume you have this [**Variable**](https://komo.do/docs/variables) already in Komodo (in Settings -> Variables)
380
+
381
+
* `MY_SECRET_1` : `aCoolSecret`
382
+
383
+
And your **Stack** config looks like this:
384
+
385
+
```
386
+
MY_NORMAL_ENV=abc
387
+
MY_API_KEY=[[MY_SECRET_1]]
388
+
```
389
+
{: file='Stack Environment Section'}
390
+
391
+
```yaml
392
+
services:
393
+
test:
394
+
image: myFakeImage
395
+
environment:
396
+
AN_ENV: ${MY_NORMAL_ENV}
397
+
API: ${MY_API_KEY}
398
+
```
399
+
{: file='compose.yaml'}
400
+
401
+
The **interpolated** `compose.yaml` will look like:
402
+
403
+
404
+
```yaml
405
+
services:
406
+
test:
407
+
image: myFakeImage
408
+
environment:
409
+
AN_ENV: abc
410
+
API: aCoolSecret
411
+
```
412
+
{: file='compose.yaml'}
413
+
414
+
The different names for all the keys in the above example are to help convey that each "section" needs to have keys defined, they aren't automatically passed. Though you CAN pass them from Komodo **Environment** section straight to container by using [`env_file:`](#ex-env-file-in-service) like `env_file: .env`
415
+
416
+
</details>
417
+
418
+
<details markdown="1">
419
+
420
+
<summary>Komodo Compose Up Flow</summary>
421
+
422
+
```mermaid
423
+
flowchart TD
424
+
425
+
Z@{ shape: text, label: "Nested Box = Docker action
426
+
Rounded box = ENV-related" }
427
+
428
+
Ko[Komodo Stack Deploy] -->|Interpolates Variables into Environment contents| Y[Final Environment Contents]
429
+
Y -->|writes to ENV File Path| A
430
+
A[[docker compose up --env-file .env + Additional Env Files section]] --> dockerenv
431
+
subgraph dockerenv [compose.yaml 'black box']
432
+
Par(["Parse <code>.env</code>, <code>--env-file</code> args, and host ENVs"]) --> B("**Interpolates** key-values from previous step into <code>${KEY_LIKE}</code> strings in compose.yaml **file only**")
433
+
end
434
+
B --> C
435
+
C[[Start service creation]] --> servicefoo
436
+
C --> servicebar
437
+
subgraph servicefoo [Service Foo]
438
+
fooenv(["Parses <code>env_file:</code> and <code>environment:</code> attributes in service Foo definition"]) --> D[[Creates Foo with ENVs from previous step **only**]]
439
+
end
440
+
subgraph servicebar [Service Bar]
441
+
barenv(["Parses <code>env_file:</code> and <code>environment:</code> attributes in service Bar definition"]) --> barser[[Creates Bar with ENVs from previous step **only**]]
442
+
end
443
+
```
444
+
445
+
</details>