2024.1.* linux/arm/v6 images fail to execute on Raspberry Pi 1 Mod. B
Closed this issue · 8 comments
Hi,
after failing to get the docker compose example to run (exited with code 0), I tried to run the command
docker run -v $PWD/cloudflared:/.cloudflared erisamoe/cloudflared:latest login
directly.
That also failed (No more output after 'Status: Downloaded ...').
So I started to try out older images and the first one working was erisamoe/cloudflared:2023.10.0.
Greetings!
Thanks for reporting I will investigate as soon as possible.
So I don't have access to a Pi 1 at the moment, the best I could try was a Pi 4 with --platform linux/arm/v6 which did work.
Click to show what was tested
erisa@cardinal:~ $ docker run --platform linux/arm/v6 -v $PWD/cloudflared:/.cloudflared erisamoe/cloudflared:latest
Unable to find image 'erisamoe/cloudflared:latest' locally
latest: Pulling from erisamoe/cloudflared
c329e0ecb505: Pull complete
d981f56a9671: Pull complete
Digest: sha256:9303e1b939bb902dcba89d8604d5ef4a66d1d211076547ece5d24303edb44123
Status: Downloaded newer image for erisamoe/cloudflared:latest
cloudflared version 2024.1.5 (built 2024-01-29-1252 UTC)
erisa@cardinal:~ $ docker run --platform linux/arm/v6 -v $PWD/cloudflared:/.cloudflared erisamoe/cloudflared:latest login
Please open the following URL and log in with your Cloudflare account:
https://dash.cloudflare.com/argotunnel?aud=&callback=https%3A%2F%2Flogin.cloudflareaccess.org%[redacted]%3D
Leave cloudflared running to download the cert automatically.
You have successfully logged in.
If you wish to copy your credentials to a server, they have been saved to:
/.cloudflared/cert.pem
erisa@cardinal:~ $ docker inspect erisamoe/cloudflared:latest
[
{
"Id": "sha256:570d3772716f049e25f244526455d4683ddb9571d87dbf8dcf1dca3884d09aae",
"RepoTags": [
"erisamoe/cloudflared:latest"
],
"RepoDigests": [
"erisamoe/cloudflared@sha256:9303e1b939bb902dcba89d8604d5ef4a66d1d211076547ece5d24303edb44123"
],
"Parent": "",
"Comment": "buildkit.dockerfile.v0",
"Created": "2024-01-29T12:55:47.737275845Z",
"Container": "",
"ContainerConfig": {
"Hostname": "",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": null,
"Cmd": null,
"Image": "",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": null,
"Labels": null
},
"DockerVersion": "",
"Author": "",
"Config": {
"Hostname": "",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"TUNNEL_ORIGIN_CERT=/etc/cloudflared/cert.pem",
"NO_AUTOUPDATE=true"
],
"Cmd": [
"version"
],
"ArgsEscaped": true,
"Image": "",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/cloudflared",
"--no-autoupdate"
],
"OnBuild": null,
"Labels": null
},
"Architecture": "arm",
"Variant": "v6",
"Os": "linux",
"Size": 34359377,
"VirtualSize": 34359377,
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/660137f1ae3e756788e6873457d545e8f471e24cde0f7e3624711af8e65e92e6/diff",
"MergedDir": "/var/lib/docker/overlay2/497fded6d90e60d02a2691428c4db40adffffbf36a258eca4ca78aaeae2f2c3b/merged",
"UpperDir": "/var/lib/docker/overlay2/497fded6d90e60d02a2691428c4db40adffffbf36a258eca4ca78aaeae2f2c3b/diff",
"WorkDir": "/var/lib/docker/overlay2/497fded6d90e60d02a2691428c4db40adffffbf36a258eca4ca78aaeae2f2c3b/work"
},
"Name": "overlay2"
},
"RootFS": {
"Type": "layers",
"Layers": [
"sha256:bd77bff50350f119d50136ba0ebc9e72fa2b189c0ca46b6f6ca76d5523ddf4be",
"sha256:a483742d85d40b698dfd880e69822015a82be70a5a65044d5627e41dcc4eec80"
]
},
"Metadata": {
"LastTagTime": "0001-01-01T00:00:00Z"
}
}
]
erisa@cardinal:~ $ ls -alh cloudflared
total 12K
drwxr-xr-x 2 root root 4.0K Feb 2 04:50 .
drwxr-xr-x 27 erisa erisa 4.0K Feb 2 04:49 ..
-rw------- 1 root root 1.9K Feb 2 04:50 cert.pem
I will keep finding a way to test in a real armv6 environment, unfortunately I don't believe I have one readily available which is why the armv6 support was listed as untested on the readme. In the meantime, do you have the full output from the command as well as docker inspect erisamoe/cloudflared:latest like the above?
Also adding for my own reference, this is likely related to the switch to Cloudflare's fork of Go, which was a necessary change to support newer versions of cloudflared: 6a40972
Here are the requested outputs:
root@rpi:~# docker run --platform linux/arm/v6 -v $PWD/cloudflared:/.cloudflared erisamoe/cloudflared:latest
Unable to find image 'erisamoe/cloudflared:latest' locally
latest: Pulling from erisamoe/cloudflared
c329e0ecb505: Pull complete
d981f56a9671: Pull complete
Digest: sha256:9303e1b939bb902dcba89d8604d5ef4a66d1d211076547ece5d24303edb44123
Status: Downloaded newer image for erisamoe/cloudflared:latest
root@rpi:# docker inspect erisamoe/cloudflared:latest
[
{
"Id": "sha256:570d3772716f049e25f244526455d4683ddb9571d87dbf8dcf1dca3884d09aae",
"RepoTags": [
"erisamoe/cloudflared:latest"
],
"RepoDigests": [
"erisamoe/cloudflared@sha256:9303e1b939bb902dcba89d8604d5ef4a66d1d211076547ece5d24303edb44123"
],
"Parent": "",
"Comment": "buildkit.dockerfile.v0",
"Created": "2024-01-29T12:55:47.737275845Z",
"Container": "",
"ContainerConfig": {
"Hostname": "",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": null,
"Cmd": null,
"Image": "",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": null,
"Labels": null
},
"DockerVersion": "",
"Author": "",
"Config": {
"Hostname": "",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"TUNNEL_ORIGIN_CERT=/etc/cloudflared/cert.pem",
"NO_AUTOUPDATE=true"
],
"Cmd": [
"version"
],
"ArgsEscaped": true,
"Image": "",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/cloudflared",
"--no-autoupdate"
],
"OnBuild": null,
"Labels": null
},
"Architecture": "arm",
"Variant": "v6",
"Os": "linux",
"Size": 34359377,
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/7a5a337f19828d78e6672634cfe2dc49ebae743e433cf76ab499ee55ddd593e4/diff",
"MergedDir": "/var/lib/docker/overlay2/81f1e31ddf3396f15c82df73d5174b7cf9aab055271d74798a9e1923dff8dda7/merged",
"UpperDir": "/var/lib/docker/overlay2/81f1e31ddf3396f15c82df73d5174b7cf9aab055271d74798a9e1923dff8dda7/diff",
"WorkDir": "/var/lib/docker/overlay2/81f1e31ddf3396f15c82df73d5174b7cf9aab055271d74798a9e1923dff8dda7/work"
},
"Name": "overlay2"
},
"RootFS": {
"Type": "layers",
"Layers": [
"sha256:bd77bff50350f119d50136ba0ebc9e72fa2b189c0ca46b6f6ca76d5523ddf4be",
"sha256:a483742d85d40b698dfd880e69822015a82be70a5a65044d5627e41dcc4eec80"
]
},
"Metadata": {
"LastTagTime": "0001-01-01T00:00:00Z"
}
}
]
root@rpi:#
Got curios so I extracted the docker container to be able to do some further investigation:
root@rpi:~/extract# ls -al
total 77952
drwxr-xr-x 7 root root 4096 Feb 3 12:39 .
drwx------ 6 root root 4096 Feb 3 12:37 ..
drwxr-xr-x 2 root root 4096 Feb 3 12:25 .cloudflared
-rwxr-xr-x 1 root root 34144868 Jan 29 13:55 cloudflared
-rw------- 1 root root 5636096 Feb 3 12:25 core
drwxr-xr-x 4 root root 4096 Feb 3 12:25 dev
-rwxr-xr-x 1 root root 0 Feb 3 12:25 .dockerenv
drwxr-xr-x 3 root root 4096 Feb 3 12:25 etc
-rw-r--r-- 1 root root 40006144 Feb 3 12:38 export.tar
drwxr-xr-x 2 root root 4096 Feb 3 12:25 proc
drwxr-xr-x 2 root root 4096 Feb 3 12:25 sys
root@rpi:~/extract# ./cloudflared
Illegal instruction (core dumped)
root@rpi:~/extract# gdb cloudflared core
GNU gdb (Raspbian 10.1-1.7) 10.1.90.20210103-git
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from cloudflared...
[New LWP 24058]
Core was generated by `./cloudflared'.
Program terminated with signal SIGILL, Illegal instruction.
#0 runtime.check () at /tmp/go/src/runtime/runtime1.go:152
152 /tmp/go/src/runtime/runtime1.go: No such file or directory.
warning: Missing auto-load script at offset 0 in section .debug_gdb_scripts
of file /root/extract/cloudflared.
Use `info auto-load python-scripts [REGEXP]' to list them.
(gdb) quit
root@rpi:~/extract#
Maybe this helps already to fix it.
Thank you! As I suspected it seems to be related to the Go change. Since that change was necessary for newer versions, you will need to keep using 2023.10.0 until I can get this fixed. Will add a note to the readme if it drags on too long.
Okay I have acquired a Pi 1 and was able to reproduce. From doing some research I found out about GOARM=6 which correctly builds for armv6. It is unclear to me why the current implementation was working until now.
Will need to add some logic to the Dockerfile to detect when it's building for armv6 and use that flag.
erisa@raspberrypi:~ $ docker run cr.erisa.uk/cloudflared:armv6 login
Please open the following URL and log in with your Cloudflare account:
You can use cr.erisa.uk/cloudflared:armv6 to test this fix if you want but I will push a proper fix to the repository soon.
@sternstauner thank you for your patience while I tracked down a Pi 1 - could you please pull the latest version of the image (or tag :2024.2.0) and try this again? It should work now, and does for me on the same hardware.
I'm also going to remove the "Untested" note from armv6 on the readme as I now have the hardware to perform manual tests.
Just updated my configuration and I can confirm that the latest image is working fine.
Thank you for quickly fixing the issue.