env2yaml not processing file in --path.confg

[willfarrell]

Would be nice if the docker-entrypoint or logstash would replace ENV. This was a feature in the now deprecated docker image.

Hack I'm doing on docker-entrypoint:

#!/bin/bash -e

# Map environment variables to entries in logstash.yml.
# Note that this will mutate logstash.yml in place if any such settings are found.
# This may be undesirable, especially if logstash.yml is bind-mounted from the
# host system.
#env2yaml /usr/share/logstash/config/logstash.yml

PATH_CONFIG=/opt/logstash/conf.d #TODO grab from args
while IFS='=' read -r name value ; do
    # replace ${name}, ${name|default}, ${name | default} => value
    find ${PATH_CONFIG} -type f -name "*.conf" -print0 | xargs -0 sed -i.bak -r 's~\$\{'${name}' ?(|\|[^\}]{1,})}~'${value}'~g'
done < <(env)
rm ${PATH_CONFIG}/*.bak
if [[ -z $1 ]] || [[ ${1:0:1} == '-' ]] ; then
  exec logstash "$@"
else
  exec "$@"
fi

[jarpy]

Hi,

Could you expand a little bit on the problem and proposed solution? I'm having trouble understanding what the goal is.

Thanks.

[willfarrell]

When using the docker container from https://hub.docker.com/_/logstash/, strings like ${ENV_ELASTICSEARCH_HOST:default_value} in the config would be replaced with their corresponding ENV in the container or fall back to the default value. I'm now migrating to this new docker image, and finding that the ENV var is not being replaced like it used too.

In your docker-entrypoint you show an example being replaced using env2yaml (env2yaml /usr/share/logstash/config/logstash.yml). This leads me to believe that this ENV replacement has been moved from inside logstash to the container. I guess I would like env2yaml to check if --path.config is set and if so, do an ENV replace on all files in that dir.

[willfarrell]

Just found your comment #24 (comment). I'll test out using ENV.PATH_CONFIG.

Edit: Seems to cause a loop where the agent keeps going over the configs.

[willfarrell]

Figured out what changed.

• key: ${ENV} -> 'value'
• key: ${ENV}/path -> None <- This use case is mentioned in the docs
• key: '${ENV}' -> ${ENV}'
• key: "${ENV}" -> ${ENV}' <- This used to work in older versions

I've reconfigured my ENV to work around this. Would be nice if "${ENV}" was supported again.