Curator running failed in docker when upgrade to 8.0.8 as permssion error

[Jae1313]

For usage questions and help

Please create a topic at https://discuss.elastic.co/c/elasticsearch

Perhaps a topic there already has an answer for you!

To submit a bug or report an issue

Expected Behavior

Actual Behavior

Steps to Reproduce the Problem

1. install curator in docker, run the curator as elasticsearch user instead of root user
   command: su elasticsearch /bin/bash -c "/usr/local/bin/curator --config /config/curator-config.yml /config/curator-actions.yml"
   curator-config.yml example:
   elasticsearch:
   client:
   hosts:
   • https://aln-tcdvc7vm-69.labs.netscout.com:9200
     cloud_id:
     ca_certs: /config/certs/cacert.pem
     client_cert:
     client_key:
     verify_certs:
     request_timeout: 30
     other_settings:
     master_only: False
     username:
     password:
     api_key:
     id:
     api_key:
     token:

logging:
loglevel: INFO
logfile: /logs/es-curator-actions.log
logformat: default
blacklist: ['elastic_transport', 'urllib3']

Command result as following error:
Traceback (most recent call last):
File "/usr/local/bin/curator", line 8, in
sys.exit(cli())
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/click/decorators.py", line 34, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/curator/cli.py", line 316, in cli
set_logging(check_logging_config(
File "/usr/local/lib/python3.8/dist-packages/curator/config_utils.py", line 51, in set_logging
loginfo = LogInfo(log_opts)
File "/usr/local/lib/python3.8/dist-packages/curator/logtools.py", line 136, in init
self.handler = logging.FileHandler('/proc/1/fd/1')
File "/usr/lib/python3.8/logging/init.py", line 1147, in init
StreamHandler.init(self, self._open())
File "/usr/lib/python3.8/logging/init.py", line 1176, in _open
return open(self.baseFilename, self.mode, encoding=self.encoding)
PermissionError: [Errno 13] Permission denied: '/proc/1/fd/1'

Specifications

• Version: curator 8.0.8
• Platform: elasticsearch 8.8.2 docker version with Ubuntu 20.04.6 LTS
• Subsystem:

Context (Environment)

Detailed Description

[untergeek]

This is clearly a container you built using Python 3.8, which means you did not use the provided Dockerfile which builds with 3.11. I can’t say for sure what’s going on here as I do not know what version of Docker you built this with, but the line that includes logging.FileHandler('/proc/1/fd/1') writes logs to the Docker socket on newer versions of Docker so that you can run docker logs CONTAINERNAME to view the logs. This works perfectly with the Curator image published to Docker hub.

Please provide more information.

UPDATE: The code will always try to log to that path when Docker is detected. The following work around is invalid. A manual workaround is to force Curator to log to a file inside the Docker image (which you can, of course, map to a local file path). This will force Curator to not log to a path usable by docker logs but to your specified path.

[Jae1313]

Which docker version have been tested to work fine? I guess maybe my docker version is a bit lower.
What I am using is:
$ docker --version
Docker version 20.10.9, build c2ea9bc

[untergeek]

It’s also the Dockerfile. You’re not using the current Dockerfile if you’re building with Python 3.8. Try building with the current Dockerfile before worrying about the Docker version, or just pull image untergeek/curator:8.0.8 and use that.

[Jae1313]

In our deployment, we do not deploy dockerized curator. What we do is deploy dockerized Elasticsearch, and install curator in Elasticsearch Dockerfile , something like:
RUN ln -s /usr/bin/python3.8 /usr/bin/python &&
pip install cryptography elasticsearch-curator

In this case, curator will still detect it as is_docker. In this scenario, I dont think it needs to write logs to the Docker socket.

[untergeek]

I can’t agree that I think this is a good idea, to be honest, precisely because of code changes like this, that turned out to be breaking changes to your setup.

I will see about putting a a write test and ignore that path if it doesn’t have permissions, but it will not be added for a two weeks at best, I would guess.

[Jae1313]

We could wait the fix then upgrade the curator to the latest one, currently our short term solution is downgrade to 8.0.4 which do not have is_docker detect.
Thanks for your support.