[BUG] unexpected character "-" in variable name since v2.29.3
Closed this issue · 7 comments
Description
Environment variables defined via an env_file that have "-" characters in their name get refused after updating docker-compose from v2.29.2
to v2.29.3
Steps To Reproduce
services:
env_runner:
image: alpine
command: env
env_file:
- ./myenv.env
myenv.env
foo-bar=test
Expected behaviour (v2.29.2
)
$ docker compose up
[+] Running 1/1
✔ Container env_runner-1 Created
Attaching to env_runner-1
[…]
env_runner-1 | foo-bar=test
env_runner-1 exited with code 0
Actual behaviour (v2.29.3
)
$ docker compose up
failed to read myenv.env: line 1: unexpected character "-" in variable name "foo-bar=test"
Compose Version
Docker Compose version 2.29.3
Docker Environment
Client:
Version: 27.2.0
API version: 1.47
Go version: go1.23.0
Git commit: 3ab4256958
Built: Thu Aug 29 16:44:26 2024
OS/Arch: linux/amd64
Context: default
Server:
Engine:
Version: 27.2.0
API version: 1.47 (minimum version 1.24)
Go version: go1.23.0
Git commit: 3ab5c7d003
Built: Thu Aug 29 16:44:26 2024
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.7.22
GitCommit: 7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c.m
runc:
Version: 1.1.14
GitCommit:
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Anything else?
No response
While technically Linux can use env-vars with hyphens as name (any character except nul
terminators), it's generally recommended to only use characters from the portable character set; https://pubs.opengroup.org/onlinepubs/7908799/xbd/envvar.html
What I suspect happened here is that the libary used by compose possibly considers them as invalid formatted substitution character, and the part after the hyphen as default
; https://tldp.org/LDP/abs/html/parameter-substitution.html
That said, env_file
for services should follow the same rules as --env-file
on docker run
(which only considers =
as separator, and performs no substitution/validation otherwise 🤔
This looks related to this PR;
However the env_file
(docker run --env-file
) should not do interpolation; is there a way to make that conditional, @ndeloof ?
docker run --rm --env "hello-world=hello-value" alpine printenv hello-world
hello-value
For reference; this is the validation in the OCI runtime (runc); https://github.com/opencontainers/runc/blob/7c2e69f1c4a7df1cf4690ccfc68c0a3857251d12/libcontainer/init_linux.go#L258-L281
// populateProcessEnvironment loads the provided environment variables into the
// current processes's environment.
func populateProcessEnvironment(env []string) error {
for _, pair := range env {
p := strings.SplitN(pair, "=", 2)
if len(p) < 2 {
return errors.New("invalid environment variable: missing '='")
}
name, val := p[0], p[1]
if name == "" {
return errors.New("invalid environment variable: name cannot be empty")
}
if strings.IndexByte(name, 0) >= 0 {
return fmt.Errorf("invalid environment variable %q: name contains nul byte (\\x00)", name)
}
if strings.IndexByte(val, 0) >= 0 {
return fmt.Errorf("invalid environment variable %q: value contains nul byte (\\x00)", name)
}
if err := os.Setenv(name, val); err != nil {
return err
}
}
return nil
}
env-file when used in compose does support interpolation (and a bunch of our users rely on this feature)
Right, but the reverse also looks to be true; it's rejecting env-vars that should be considered valid 🤔
Environment variable names used by the utilities in the Shell and Utilities volume of IEEE Std 1003.1-2001 consist solely of uppercase letters, digits, and the '_' (underscore) from the characters defined in Portable Character Set and do not begin with a digit. Other characters may be permitted by an implementation;
So -
MAY be permitted, and I can guess some will rely on it. To get actual support of this, we will have to make the dotenv parser way more flexible that it is 😓
Yeah, I recall we once added validation for environment-variables, but then had to revert, for similar reasons;
When will the fix be published? I’ve tested docker compose v2.29.7
and I still get the same error.