zebrium/ze-kubernetes-collector

zlog-collector requires /etc/localtime

Opened this issue · 1 comments

suvl commented

Well, I have some machines running RancherOS, and they don't have /etc/localtime as shown:

Events:
  Type     Reason     Age                  From                    Message
  ----     ------     ----                 ----                    -------
  Normal   Scheduled  <unknown>            default-scheduler       Successfully assigned zebrium/zlog-collector-4hs64 to docker-vmss-1
  Warning  Failed     2m21s                kubelet, docker-vmss-1  Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/etc/localtime\\\" to rootfs \\\"/var/lib/docker/overlay2/2087d8e3a8d2db5dcc669407a2879c028c83157b0875ff88d7d9feb66626ff82/merged\\\" at \\\"/var/lib/docker/overlay2/2087d8e3a8d2db5dcc669407a2879c028c83157b0875ff88d7d9feb66626ff82/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
  Warning  Failed     2m21s                kubelet, docker-vmss-1  Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/etc/localtime\\\" to rootfs \\\"/var/lib/docker/overlay2/8e48fe511efa2e4ed858e3457865f2a0d60e15101ba6d416da69f08b383fd4c8/merged\\\" at \\\"/var/lib/docker/overlay2/8e48fe511efa2e4ed858e3457865f2a0d60e15101ba6d416da69f08b383fd4c8/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
  Warning  Failed     2m5s                 kubelet, docker-vmss-1  Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/etc/localtime\\\" to rootfs \\\"/var/lib/docker/overlay2/1d3cb742ce8715861be2649fab32730508ff658c0829bc7446da53e6b1a697be/merged\\\" at \\\"/var/lib/docker/overlay2/1d3cb742ce8715861be2649fab32730508ff658c0829bc7446da53e6b1a697be/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
  Warning  Failed     98s                  kubelet, docker-vmss-1  Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/etc/localtime\\\" to rootfs \\\"/var/lib/docker/overlay2/c99a4e457ebe5fdebb108441220a14676976444624fbe94dc13e663fb24f4af3/merged\\\" at \\\"/var/lib/docker/overlay2/c99a4e457ebe5fdebb108441220a14676976444624fbe94dc13e663fb24f4af3/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
  Normal   Created    52s (x5 over 2m22s)  kubelet, docker-vmss-1  Created container zlog-collector
  Normal   Pulled     52s (x5 over 2m22s)  kubelet, docker-vmss-1  Container image "zebrium/zlog-collector:latest" already present on machine
  Warning  Failed     52s                  kubelet, docker-vmss-1  Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/etc/localtime\\\" to rootfs \\\"/var/lib/docker/overlay2/74e0367cc231875c78b83c77a31895dc0cb48a18fe6730a20dc40e2e8d9d0f1f/merged\\\" at \\\"/var/lib/docker/overlay2/74e0367cc231875c78b83c77a31895dc0cb48a18fe6730a20dc40e2e8d9d0f1f/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
  Warning  BackOff    11s (x9 over 2m20s)  kubelet, docker-vmss-1  Back-off restarting failed container

trying to mount directly my timezone also won't work:

Events:
  Type     Reason     Age                   From                    Message
  ----     ------     ----                  ----                    -------
  Normal   Scheduled  <unknown>             default-scheduler       Successfully assigned zebrium/zlog-collector-l5qt4 to docker-vmss-1
  Warning  Failed     14m                   kubelet, docker-vmss-1  Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/usr/share/zoneinfo/Europe/Lisbon\\\" to rootfs \\\"/var/lib/docker/overlay2/dbd01800fd4de6382c806dc1c950cc0c99cbfa9085822a61fc8b8c984390bd0d/merged\\\" at \\\"/var/lib/docker/overlay2/dbd01800fd4de6382c806dc1c950cc0c99cbfa9085822a61fc8b8c984390bd0d/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
  Warning  Failed     14m                   kubelet, docker-vmss-1  Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/usr/share/zoneinfo/Europe/Lisbon\\\" to rootfs \\\"/var/lib/docker/overlay2/a9906d9d3957a8f653ac97a92173f10a8c3bb5fcd02eb3b2bed8f5cb657a79b6/merged\\\" at \\\"/var/lib/docker/overlay2/a9906d9d3957a8f653ac97a92173f10a8c3bb5fcd02eb3b2bed8f5cb657a79b6/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
  Warning  Failed     14m                   kubelet, docker-vmss-1  Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/usr/share/zoneinfo/Europe/Lisbon\\\" to rootfs \\\"/var/lib/docker/overlay2/3a772d8cb92cedde9c6594dbb0f8a2242b42b37f10253087f762a075eb8d19f2/merged\\\" at \\\"/var/lib/docker/overlay2/3a772d8cb92cedde9c6594dbb0f8a2242b42b37f10253087f762a075eb8d19f2/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
  Warning  Failed     14m                   kubelet, docker-vmss-1  Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/usr/share/zoneinfo/Europe/Lisbon\\\" to rootfs \\\"/var/lib/docker/overlay2/ac9e09ba373401fedd924586fd9fd5ae9a47bc93790057f6d7b47dd36635fcad/merged\\\" at \\\"/var/lib/docker/overlay2/ac9e09ba373401fedd924586fd9fd5ae9a47bc93790057f6d7b47dd36635fcad/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
  Normal   Created    13m (x5 over 14m)     kubelet, docker-vmss-1  Created container zlog-collector
  Normal   Pulled     13m (x5 over 14m)     kubelet, docker-vmss-1  Container image "zebrium/zlog-collector:latest" already present on machine
  Warning  Failed     13m                   kubelet, docker-vmss-1  Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/usr/share/zoneinfo/Europe/Lisbon\\\" to rootfs \\\"/var/lib/docker/overlay2/00dbf0e393b4c806d0a70f28bfeeed209817a7a7a73ebdecf15e5ec3d666efd9/merged\\\" at \\\"/var/lib/docker/overlay2/00dbf0e393b4c806d0a70f28bfeeed209817a7a7a73ebdecf15e5ec3d666efd9/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
  Warning  BackOff    4m49s (x45 over 14m)  kubelet, docker-vmss-1  Back-off restarting failed container

I think there might be no tzdata on this OS, so it might be helpful to have a way to select it by env var or something.

My solution ATM was to add an initContainer linking the expected timezone:

      initContainers:
      - name: tzdata-fixer
        image: busybox
        imagePullPolicy: IfNotPresent
        args:
        - ln
        - -fs
        - /usr/share/zoneinfo/Europe/Lisbon
        - /etc/localtime

Hi Joao, thanks for raising the issue.

It appears there is no good way to automate this except using an environment variable. We can provide a timezone config parameter in helm chart. If the config parameter is set, /etc/localtime will not be mounted on log collector container.

  • Brady