pguyot/arm-runner-action

No networking inside DietPi image

bvobart opened this issue · 8 comments

bvobart commented

Hi there. I'm trying to make a custom DietPi image with some pre-installed software, but apparently there is no network connectivity during the commands execution. How do I fix this?

Reproducible Example

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: pguyot/arm-runner-action@v2
        id: build_image
        with:
          base_image: dietpi:rpi_armv8_bullseye
          cpu: cortex-a53
          image_additional_mb: 500
          commands: |
            ping -4nc 1 -W 10 9.9.9.9

Expected output

I expect the ping to succeed and thus show that there is network connectivity in the emulated DietPi image during commands execution.

Actual output

 PING 9.9.9.9 (9.9.9.9) 56(84) bytes of data.

--- 9.9.9.9 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms
pguyot commented

As exhibited by new test, ping doesn't work in Github Action (this is not a limit of this action) but wget does work, so you can install software. Please feel free to reopen if the problem remains.

  • 👍1
bvobart commented

Ahh yes, having just Googled it, ping indeed doesn't work in GitHub Actions because it's hosted in Azure, where it doesn't work by design.

However, dietpi-software as bundled in the DietPi image does require ping to work for checking network connectivity before being able to install software.

Here's the relevant part from the logs of my CI build:

Click to view the logs 
+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/boot/dietpi
+ dietpi-software install 5 6 7 170 17 188 67
[ .... ] DietPi-Software | Initialising database
/boot/dietpi/dietpi-software: line 321: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 348: ((: == 1 &&  > 6 : syntax error: operand expected (error token is "== 1 &&  > 6 ")
/boot/dietpi/dietpi-software: line 493: ((: == 10 : syntax error: operand expected (error token is "== 10 ")
/boot/dietpi/dietpi-software: line 502: ((: == 1 ||  == 11 : syntax error: operand expected (error token is "== 1 ||  == 11 ")
/boot/dietpi/dietpi-software: line 525: ((: == 1 &&  > 6 : syntax error: operand expected (error token is "== 1 &&  > 6 ")
/boot/dietpi/dietpi-software: line 593: ((: == 10 &&  > 6 : syntax error: operand expected (error token is "== 10 &&  > 6 ")
/boot/dietpi/dietpi-software: line 677: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/boot/dietpi/dietpi-software: line 687: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/boot/dietpi/dietpi-software: line 698: ((: == 10 : syntax error: operand expected (error token is "== 10 ")
/boot/dietpi/dietpi-software: line 700: ((: == 2 : syntax error: operand expected (error token is "== 2 ")
/boot/dietpi/dietpi-software: line 709: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/boot/dietpi/dietpi-software: line 766: ((: > 6 : syntax error: operand expected (error token is "> 6 ")
/boot/dietpi/dietpi-software: line 792: ((: > 6 : syntax error: operand expected (error token is "> 6 ")
/boot/dietpi/dietpi-software: line 887: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 898: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 922: ((: == 2 : syntax error: operand expected (error token is "== 2 ")
/boot/dietpi/dietpi-software: line 951: ((: == 2 : syntax error: operand expected (error token is "== 2 ")
/boot/dietpi/dietpi-software: line 969: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 979: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1051: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1055: ((: < 7 : syntax error: operand expected (error token is "< 7 ")
/boot/dietpi/dietpi-software: line 1098: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1192: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1200: ((: > 19 : syntax error: operand expected (error token is "> 19 ")
/boot/dietpi/dietpi-software: line 1209: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1216: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1225: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1233: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1255: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1264: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1446: ((: == 75 : syntax error: operand expected (error token is "== 75 ")
/boot/dietpi/dietpi-software: line 1506: ((: == 3 ||  == 10 ||  > 6 : syntax error: operand expected (error token is "== 3 ||  == 10 ||  > 6 ")
/boot/dietpi/dietpi-software: line 1516: ((: < 10 &&  > 6 : syntax error: operand expected (error token is "< 10 &&  > 6 ")
[  OK  ] DietPi-Software | Initialised database
 DietPi-Software
─────────────────────────────────────────────────────
 Step: Applying initial first run setup steps
[ .... ] DietPi-Software | Checking IPv4 network connectivity (1/2)
[ .... ] DietPi-Software | Checking IPv4 network connectivity (2/2)
[FAILED] DietPi-Software | Checking IPv4 network connectivity
 - Command: ping -4nc 1 -W 10 IP

---------------------------------------------------------------------
- DietPi has encountered an error                                   -
- Please create a ticket: https://github.com/MichaIng/DietPi/issues -
- Copy and paste only the BLUE lines below into the ticket          -
---------------------------------------------------------------------
#### Details:
- Date           | Tue Apr 30 21:42:34 UTC 2024
- DietPi version | v9.3.0 (MichaIng/master)
- Image creator  | 
- Pre-image      | 
- Hardware       |  (ID=)
- Kernel version | `Linux fv-az1210-406 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 aarch64 GNU/Linux`
- Distro         |  (ID=)
- Command        | `ping -4nc 1 -W 10 IP`
- Exit code      | 1
- Software title | DietPi-Software
#### Steps to reproduce:
<!-- Explain how to reproduce the issue -->
1. ...
2. ...
#### Expected behaviour:
<!-- What SHOULD happen? -->
- ...
#### Actual behaviour:
<!-- What IS happening? -->
- ...
#### Extra details:
<!-- Please post any extra details that might help solve the issue -->
- ...
#### Additional logs:
```
PING 9.9.9.9 (9.9.9.9) 56(84) bytes of data.
--- 9.9.9.9 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms
```
---------------------------------------------------------------------
[FAILED] DietPi-Software | Unable to continue, DietPi-Software will now terminate.
Error: Process completed with exit code 130.

So then it seems like it won't be possible to install software in a DietPi image with dietpi-software without running it on a self-hosted CI runner where ping is available?

EDIT: I've been toying around a bit trying to create a Dockerfile for a GitHub Actions runner image that I can run on my desktop PC that will work with this ARM runner action, but I can't seem to get it to work yet due to losetup failing. Do you have any tips or pointers to how I can get this working?

pguyot commented

I never tried to run this action in a self-hosted runner image, and there is issue #96 which is related to a GitHub bug and potential fix #102.

Meanwhile, maybe you can patch dietpi-software to avoiding pinging or install a fake ping binary to fool it. @MichaIng may have a clue.

  • 👍1
MichaIng commented

For our own builds and tests, we set CONFIG_CHECK_CONNECTION_IP=127.0.0.1 in dietpi.txt as a workaround for GitHub hosted runners. And in case it shall become an image, don't forget to reset it to e.g. 9.9.9.9 (our default). There is also CONFIG_CHECK_CONNECTION_IPV6, but GitHub is IPv4-only, so that one can be left unchanged. Probably an automatic fallback to HTTP/curl would be a good idea for cases where system or network does not support ICMP.

Out of interest. You use this action to generate a DietPi image with software preinstalled? In case, good that we have a plan to implement this ability right into our dietpi-installer and dietpi-build scripts. It will take some time, as quite some rework is required, but keep an eye on release notes.

  • 👍1
  • ❤️1
bvobart commented

For our own builds and tests, we set CONFIG_CHECK_CONNECTION_IP=127.0.0.1 in dietpi.txt as a workaround for GitHub hosted runners. And in case it shall become an image, don't forget to reset it to e.g. 9.9.9.9 (our default).

Niice thanks, that works for me too! dietpi-software is now able to install software 🥳

Indeed, it's becoming an image so I've made sure to reset it to 9.9.9.9 at the end of my build script.

Out of interest. You use this action to generate a DietPi image with software preinstalled? In case, good that we have a plan to implement this ability right into our dietpi-installer and dietpi-build scripts. It will take some time, as quite some rework is required, but keep an eye on release notes.

Yes! I'm developing Pi-DJ in which I'm building a DietPi image with everything pre-installed and configured for turning a Raspberry Pi with touchscreen and a MIDI DJ deck into a standalone portable DJ setup. In the build script I'm running dietpi-software to pre-install X.Org, Alsa, ffmpeg, and a couple other packages onto the image.

That sounds like an interesting feature, but I'm not sure I understand yet how it will benefit a project like mine. There's some other stuff I install and configure too, so I still need to run a custom build script in a runner action like this and a single dietpi-software install .... line in there is quite easy to handle :)

I'll still keep an eye out for the release notes though, thanks for your reply and efforts in the open-source community!

bvobart commented

@MichaIng Btw, do you happen to know what is causing all the shell scripting syntax errors that I see in my build logs while running dietpi-software? I'm not sure if it's breaking stuff because it doesn't crash or anything but I can only assume something must be breaking :P

Here's an example:

[ .... ] DietPi-Software | Initialising database
/boot/dietpi/dietpi-software: line 321: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 348: ((: == 1 &&  > 6 : syntax error: operand expected (error token is "== 1 &&  > 6 ")
/boot/dietpi/dietpi-software: line 493: ((: == 10 : syntax error: operand expected (error token is "== 10 ")
/boot/dietpi/dietpi-software: line 502: ((: == 1 ||  == 11 : syntax error: operand expected (error token is "== 1 ||  == 11 ")
/boot/dietpi/dietpi-software: line 525: ((: == 1 &&  > 6 : syntax error: operand expected (error token is "== 1 &&  > 6 ")
/boot/dietpi/dietpi-software: line 593: ((: == 10 &&  > 6 : syntax error: operand expected (error token is "== 10 &&  > 6 ")
/boot/dietpi/dietpi-software: line 677: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/boot/dietpi/dietpi-software: line 687: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/boot/dietpi/dietpi-software: line 698: ((: == 10 : syntax error: operand expected (error token is "== 10 ")
/boot/dietpi/dietpi-software: line 700: ((: == 2 : syntax error: operand expected (error token is "== 2 ")
/boot/dietpi/dietpi-software: line 709: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/boot/dietpi/dietpi-software: line 766: ((: > 6 : syntax error: operand expected (error token is "> 6 ")
/boot/dietpi/dietpi-software: line 792: ((: > 6 : syntax error: operand expected (error token is "> 6 ")
/boot/dietpi/dietpi-software: line 887: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 898: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 922: ((: == 2 : syntax error: operand expected (error token is "== 2 ")
/boot/dietpi/dietpi-software: line 951: ((: == 2 : syntax error: operand expected (error token is "== 2 ")
/boot/dietpi/dietpi-software: line 969: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 979: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1051: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1055: ((: < 7 : syntax error: operand expected (error token is "< 7 ")
/boot/dietpi/dietpi-software: line 1098: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1192: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1200: ((: > 19 : syntax error: operand expected (error token is "> 19 ")
/boot/dietpi/dietpi-software: line 1209: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1216: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1225: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1233: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1255: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1264: ((: > 9 : syntax error: operand expected (error token is "> 9 ")
/boot/dietpi/dietpi-software: line 1446: ((: == 75 : syntax error: operand expected (error token is "== 75 ")
/boot/dietpi/dietpi-software: line 1506: ((: == 3 ||  == 10 ||  > 6 : syntax error: operand expected (error token is "== 3 ||  == 10 ||  > 6 ")
/boot/dietpi/dietpi-software: line 1516: ((: < 10 &&  > 6 : syntax error: operand expected (error token is "< 10 &&  > 6 ")
[  OK  ] DietPi-Software | Initialised database

It seems like all of those syntax errors revolve around comparisons with empty variables. I hope it's a simple thing and you know of a quick fix. Otherwise, should I make an issue on the DietPi repo to continue figuring it out there?

MichaIng commented

Hmm, indeed G_HW_ARCH and G_HW_MODEL seem to be not defined. I guess the problem is, that the image has not been "booted", so /boot/dietpi/preboot did not run, which creates /boot/dietpi/.hw_model, which contains those variables.

Please do the following in your script, before running dietpi-software:

/boot/dietpi/func/dietpi-obtain_hw_model

The exact RPi model is detected based on /proc/cpuinfo. If it does not exist, or does not contain known RPi revision codes, it will assume RPi 1. You can either use the cpu_info input of this action to mount one of the bundled cpuinfo files, or apply e.g. G_HW_MODEL=4 for RPi 4 to mentioned file, to change this:

sed -i '/^G_HW_MODEL=/c\G_HW_MODEL=4' /boot/dietpi/.hw_model

This affects a few software settings, nothing critical (some cache sizes, parallel download limits and such), but if the image is for RPi 4 explicitly, it won't hurt to make dietpi-software treat it like that.

/boot/dietpi/func/dietpi-obtain_hw_model runs on every regular boot, so even if the image is booted on another RPi model later, for any future installs, dietpi-config etc, adding above to the image build script won't have a persistent effect.

  • ❤️1
bvobart commented

You're correct, the image has indeed not been 'booted' and running /boot/dietpi/func/dietpi-obtain_hw_model was the trick to ensuring the variables you mention exist.

Thanks for your support and further explanation!!

  • 👍1