The behavior of the --env option changed in CLI 8.2+

[stevenharman]

The --env= option not behaving as expected. Or, at least not the way it used to. (⬅️ Minimal reproduction code mentioned here, is in that Gist.)

This minimal reproduction

In this example repo, you'll see a "base" .env file which sets BASE_ENV_VAR="value for base" and has a blank OTHER_ENV_VAR=.
The .env.local file declares OTHER_ENV_VAR="value for other".
The .env.empty is just an empty file, with no content.

Finally the Procfile has a single entry that runs the read_some_env_vars Ruby executable script.
That script just prints out the values of ENV["BASE_ENV_VAR"] and ENV["OTHER_ENV_VAR"].

Running the executable Ruby script directly results in exactly what we'd expect, both values being not_set:

$ ./read_some_env_vars
⚠️
BASE_ENV_VAR=not_set
OTHER_ENV_VAR=not_set
⚠️

Then using heroku local:run we would expect to see the values from the .env, and we do:

$ heroku local:run ./read_some_env_vars
[OKAY] Loaded ENV .env File as KEY=VALUE Format
⚠️
BASE_ENV_VAR=value for base
OTHER_ENV_VAR=
⚠️

What's wrong?

In the Heroku CLI prior to 8.2, passing --env=/dev/null (or any empty file) as the Environment file would result in nothing additional being loaded into the ENV.
We would do this because we need to load multiple files, in a certain order, to match the behavior of dotenv-rails's loading.

Specifically, we want to load .env.local (ignored from our Git repo) first, and then .env (checked into Git).
The .env hold base configuration needed, and a number of blank values with comments explaining what these are used for.
Then the ignored-from-Git .env.local overrides some of the entries from the .env file.

Using our earlier example, we'll now send --env=/dev/null and then --env=.env.empty.
In both cases we'd expect to see output similar to directly running the read_some_env_vars script - not_set for both.
But… instead we see the values from the .env file:

$ heroku local:run ./read_some_env_vars --env=/dev/null
[OKAY] Loaded ENV /dev/null File as KEY=VALUE Format
⚠️
BASE_ENV_VAR=value for base
OTHER_ENV_VAR=
⚠️

$ heroku local:run ./read_some_env_vars --env=.env.empty
[OKAY] Loaded ENV .env.empty File as KEY=VALUE Format
⚠️
BASE_ENV_VAR=value for base
OTHER_ENV_VAR=
⚠️

If we use the dotenv executable to load .env.local before we run heroku local, we would only expect to see BASE_ENV_VAR as not_set, with OTHER_ENV_VAR having the value from .env.local.
Again, we don't get what we'd expect.
We do see the OTHER_ENV_VAR with the expected value from .env.local, so that's good!
But we also see the BASE_ENV_VAR value from .env.
And that's bad.

$ dotenv -f .env.local heroku local:run ./read_some_env_vars --env=/dev/null
[OKAY] Loaded ENV /dev/null File as KEY=VALUE Format
⚠️
BASE_ENV_VAR=value for base
OTHER_ENV_VAR=value for other
⚠️

Finally, if we tell heroku local to explicitly load just the .env.local, we see it do that, and also load the .env.
Which it should not!

$ heroku local:run ./read_some_env_vars --env=.env.local
[OKAY] Loaded ENV .env.local File as KEY=VALUE Format
⚠️
BASE_ENV_VAR=value for base
OTHER_ENV_VAR=value for other
⚠️

Conclusion

Something changed from Heroku CLI 8.1.9 to 8.2+ wherein the .env file is being loaded regardless of the list of files given via the --env argument.

[k80bowman]

@stevenharman Thank you so much for the detailed description of the issue. I will look into this today.

[k80bowman]

@stevenharman we are still working on a fix for this bug right now, but it is going to take us a bit. In the meantime, the workaround is to use version 8.1.9. If you need to install this locally you can use heroku update --version=8.1.9. We will update this issue as soon as we release a fix. Thank you for your patience.

[stevenharman]

@k80bowman Thanks for the workaround. We're looking forward to getting the fix. Thank you.

[stevenharman]

Hello @k80bowman, and thank you, again.

We've used the heroku update --version=8.1.9 workaround to downgrade successfully. Unfortunately the standalone (and brew-installed) CLI eventually auto-upgrades and quietly breaks the usage again.

There used to be a way to disable auto-updates (back when the CLI was written in Golang), but I think that was lost with the move to OClif. So now our devs bump into this on a pretty regular basis.

I was wondering if y'all had any news/updates on getting this underlying .env treatment fixed so we don't also have to fight the auto-updating battle.

Thank you.

[k80bowman]

@stevenharman I merged a fix today (it autoclosed the issue, sorry). I'm going to reopen this until we can get a release out and confirm that everything is working properly for you.

[k80bowman]

@stevenharman this issue should be fixed in version 8.6.0. I'm closing this issue for now, please feel free to reopen it if you continue to experience problems.

[stevenharman]

Thank you, @k80bowman. I've just confirmed this is fixed on 8.6.0. Hopefully the Homebrew tap is updated soon as well.

💜