deviantony/docker-elk

Error mounting configuration files

abeoliver opened this issue · 4 comments

abeoliver commented

Problem description

I cloned the git repo from main to a folder in my home directory. I ran docker compose build and it worked with no errors. However, when I run docker compose up setup, I receive the following error:

WARN[0000] mount of type `volume` should not define `bind` option 
[+] Running 4/0
 ⠿ Network docker-elk_elk                Created  0.0s
 ⠿ Volume "docker-elk_elasticsearch"     Created   0.0s
 ⠿ Container docker-elk-elasticsearch-1  Created  0.0s
 ⠿ Container docker-elk-setup-1          Recreated    0.0s
Attaching to docker-elk-setup-1
Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/Users/username/git/docker-elk/elasticsearch/config/elasticsearch.yml" to rootfs at "/usr/share/elasticsearch/config/elasticsearch.yml": mount /Users/username/git/docker-elk/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml (via /proc/self/fd/6), flags: 0x5001: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type

Extra information

Attempts to solve:

  • I looked into fixing this issue with fixing permissions like #861
  • I tried cloning the repo to the /tmp folder instead of /home
  • I tried re-writing the docker-compose file with long-syntax volume mounts

However, I was able to achieve success in building and running the stack with a workaround. If I remove all of the volume mounts (except for the elasticsearch data volume created by docker-compose.yaml:52/113) and use COPY statements in each Dockerfile to transfer the configuration files to each container, the stack starts up with no issues. Note: required 3cpu/6gb memory on colima because 2cpu/4gb was causing issues.

Stack configuration

No changes to the repo. Running Colima (v0.5.5) on an ARM M1 Macbook (MacOs 13.4.1).

Docker setup

$ docker version

Client: Docker Engine - Community
 Version:           23.0.1
 API version:       1.41 (downgraded from 1.42)
 Go version:        go1.19.5
 Git commit:        a5ee5b1dfc
 Built:             Thu Feb  9 19:15:59 2023
 OS/Arch:           darwin/arm64
 Context:           colima

Server:
 Engine:
  Version:          20.10.18
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.6
  Git commit:       e42327a6d3c55ceda3bd5475be7aae6036d02db3
  Built:            Sun Sep 11 07:10:00 2022
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          v1.6.8
  GitCommit:        9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
 runc:
  Version:          1.1.4
  GitCommit:        5fd4c4d144137e991c4acebb2146ab1483a97925
 docker-init:
  Version:          0.19.0
  GitCommit:        

$ docker-compose version

Docker Compose version 2.20.2

Container logs

$ docker-compose logs

docker-elk-kibana-1  | [2023-08-09T01:28:01.478+00:00][INFO ][node] Kibana process configured with roles: [background_tasks, ui]
docker-elk-logstash-1  | Using bundled JDK: /usr/share/logstash/jdk
antoineco commented

This is a very interesting error. (The warning on the first line is harmless, you can ignore it.)

I have a feeling that Lima's sshfs might be the culprit here. Possibly the files on your host were not fully synced into the VM?

  1. If you ssh into it, is /Users/username/git/docker-elk/ populated?
  2. If yes, what does file elasticsearch/elasticsearch.yml return inside that directory?

I am not convinced of that theory myself, because the Compose file is obviously there, and you are able to build images that include the config files.

What is the guest OS inside the Lima VM?

abeoliver commented

Thanks @antoineco for getting back to me so quickly!

The only file in /Users/username/ was Library/Caches/colima/. This does suggest that something is not being mounted correctly but, as you mentioned, I am still able to do other tasks that should also require a mounted fs. I understand if you believe that this is out of scope but I appreciate your help nonetheless.

In some more testing I found that:

  • If, in docker-compose.yml, I mount ./elasticsearch/config to a random directory instead of mounting ./elasticsearch/config/elasticsearch.yml to /usr/share/elasticsearch/config/elasticsearch.yml, the error goes away
  • If I mount ./elasticsearch/config to /usr/share/elasticsearch/config (which i realize would break things later), the error goes away
  • If I touch /tmp/a.yml in the Dockerfile and mount ./elasticsearch/config/elasticsearch.yml to /tmp/a.yml, the error persists

Here is my lima VM info:

❯ colima ssh

colima:~$ uname -a
Linux colima 5.15.68-0-virt #1-Alpine SMP Fri, 16 Sep 2022 06:29:31 +0000 aarch64 Linux

colima:~$ cat /etc/os-release 
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.16.2
PRETTY_NAME="Alpine Linux v3.16"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://gitlab.alpinelinux.org/alpine/aports/-/issues"
BUILD_ID=""
VARIANT_ID="clm"
antoineco commented

The only file in /Users/username/ was Library/Caches/colima/. This does suggest that something is not being mounted correctly

Ah, there you go! Mounting files inside containers requires them to be present on the Docker host. Copying files into images, on the other hand, is done by copying the build context over a client-server communication, which is why this approach worked for you.

I've had a very good experience working with Lima myself, also for working with containers. I don't know what Colima is adding on top, but adding a mount for your home directory to your Lima definition should allow you to resolve this situation.

abeoliver commented

Yes, that fixes it! It seems as though I had some non-standard mounts from an old project left behind. Thank you for your help!!

  • 👍1