Question: Using docker secrets with swarm launcher?

[shrikeh]

I was wondering given that the underlying container is run via docker-compose and not swarm, is it possible that I give the launcher a secret and it interprets it for the underlying container? For example, I'm looking at the gluetun vpn client and it owuld be great if the environment variables came from a secret (indirectly, with the swarm launcher interpreting it as env vars)

[tlex]

This is an interesting question.

This feature doesn't exist as such, but there might be a way to make it work.

To check that I understand the use case better, I'll try structuring it a bit:

• you store a USER and a PASS as docker swarm secrets
• you start swarm-launcher with those secrets exposed inside (let's call them /run/secrets/user' and /run/secrets/pass')
• you start gluetun:latest with a volume mount /run/secrets/user:/run/secrets/user and another one /run/secrets/user:/run/secrets/pass

Does that cover it?

If yes, it's possible to use the environment variable LAUNCH_VOLUMES which, in our example, would be set to:

secrets:
  user:
    external: true # assuming that you manage storing the secrets outside of the stack file
  pass:
    external: true
services:
  swarm-launcher-gluetun:
    secrets:
      - user
      - pass
    image: ghcr.io/ix-ai/swarm-launcher:latest
    [...]
    environment:
      LAUNCH_IMAGE: 'qmcgaw/gluetun:latest' # Note: I don't know gluetun, I chose the first image from google
      LAUNCH_ENVIRONMENTS: 'OPENVPN_PASSWORD_SECRETFILE=/run/secrets/pass OPENVPN_USER_SECRETFILE=/run/secrets/user'
      LAUNCH_VOLUMES: '/run/secrets/pass:/run/secrets/pass:ro /run/secrets/user:/run/secrets/user:ro' # setting to `ro` since no write access is needed
      [...]

Hope this helps

[shrikeh]

It looks like a sensible approach and means that the LAUNCH_ENVIRONMENTS section won't have to be as exhaustive as I was dreading. I'm going to have a play based on the above and come back to you. If it's successful, I can attempt a PR of documentation, because I think a lot of people would like the slickness of a swarm node being able to action swarm secrets to something like a VPN container

[shrikeh]

As a tweak to the above, is there any way to also provide the underlying container a .env file for the less sensitive environment variables? Or perhaps a template? I'm happy to have a go at a PR if this functionality isn't present but desirable.

[tlex]

The --envfile option isn't currently supported, but the workaround would be to mount a file in the swarm service and to expose it as /.env

From the documentation:

You can set default values for environment variables using a .env file, which Compose automatically looks for in project directory (parent folder of your Compose file). Values set in the shell environment override those set in the .env file.

Later edit: I have to say, this is untested. If it indeed looks for .env in the parent folder, since there isn't any parent folder for root (/), it might fail. In this case, I'll look into adding LAUNCH_ENVFILE_PATH to swarm-launcher.

[shrikeh]

Thanks, sorry I missed that, I apologise - I always try and find something in docs first before bothering maintainers.

Feedback on the secrets so far:

gluetun appears to be failing with:

ERROR reading from secret files: cannot read OpenVPN settings: cannot read user file: read /run/secrets/openvpn_user: is a directory which I assume is that there is some docker secret magic like ecryptfs or similar going on.

[tlex]

Alright, I now understand what happens. The swarm-launcher doesn't start a separate docker daemon, but connects to the one on the host. So, when specifying the volumes, they need to be paths present on the host.

Now, docker compose doesn't have access to docker swarm secrets. The only possible workaround I can think of, is to have swarm-launcher read the secrets exposed to it and then add them as environment variables to docker compose - since you don't really want to create files with the secrets on the host. Which means, the whole reason to use secrets is moot, since the private information would be easily visible on the host (either with docker inspect or with cat /proc/$pid/environ).

[tlex]

Concerning .env, once this pipeline is done (https://gitlab.com/ix.ai/swarm-launcher/-/pipelines/686452378) you can take the dev image for a spin (see if it works as expected): ghcr.io/ix-ai/swarm-launcher:dev-branch.

[tlex]

I must say, this was a bit more complicated than expected. I've now finished adding two additional variables:

• LAUNCH_ARG_ENVFILE
• LAUNCH_ENVFILES

I've updated the documentation for the two of them as well:

Variable	Default	Mandatory	Description
LAUNCH_ENVFILES	-	NO	Space separated list of Key=Value pairs. Note: These files must be present on the host where the container is started
LAUNCH_ARG_ENVFILE	-	NO	The path inside the swarm-launcher container with the env file used by docker compose --env-file XXX up

If you find them useful, I would be more than happy to accept any PR for the documentation, since the small "workaround" project has grown to be quite big in the meanwhile.

[tlex]

As stated before, it's not yet merged to master. To try it out, you'll need to use the dev-branch (unstable) tag.

[shrikeh]

Wow, you had a productive day (or evening, who knows with timezones). I will go take a look!

[shrikeh]

We are through, the vpn client appears to be spun up and connecting! Thanks for your help and yes once I have got a little further in my swarm setup using the container I would be happy to contribute

[tlex]

Thank you for your feedback. I'll merge later today the changes and create a new release.

[tlex]

Done in v0.17.0 (527275f)