crazy-max/ghaction-docker-buildx

[Q] How can I apply a workaround for "32bit build within a 64bit host using qemu"?

iav opened this issue · 3 comments

iav commented

There are a but "32bit build within a 64bit host using qemu".
More details and workaround here uraimo/run-on-arch-action#9, rust-lang/cargo#7451https://lkml.org/lkml/2018/12/28/461

In my case I need to have a rust builder container for armv7 target have tmpfs or other 32/64 bit error-free fs for /usr/local/cargo/registry and, maybe, /root/.cargo/ folders.

I it possible to apply something like

mkdir ~/.cargo
mount -t tmpfs -o size=2048m tmpfs ~/.cargo

for all builder container?

@iav I need more info. Please create a proper bug report.

iav commented

It's not a bug. At least — not your program bug, that can be "fixed".
It complicated behaviour between qemu emulated 32 bit on 64 bit host when used 64 bit inode numbers of filesystem and they try to pass into guest system. Sorry, I can tell something wrong, it's my understanding of other related links I read about it.

It's hard to reproduce, but very often occurs on wps hostings like aws, azure etc.

For example — I got it when I try to run your action, and got that situation:
https://github.com/iav/lemmy/runs/630927408?check_suite_focus=true#step:6:813

Note line 813:

#86 [linux/arm/v7 docs 4/5] RUN cargo install mdbook
#86 2.331     Updating crates.io index
#86 3.312 warning: spurious network error (2 tries remaining): could not read directory '/usr/local/cargo/registry/index/github.com-1ecc6299db9ec823/.git//refs': Value too large for defined data type; class=Os (2)
#86 3.408 warning: spurious network error (1 tries remaining): could not read directory '/usr/local/cargo/registry/index/github.com-1ecc6299db9ec823/.git//refs': Value too large for defined data type; class=Os (2)
#86 3.484 error: failed to fetch `https://github.com/rust-lang/crates.io-index`
#86 3.486 

Don't trust this "spurious network error" — there nothing about network.

In case this error occures "on local machine" — it can be solved providing smaller filesystem, where 64 bit inodes will not use, like

dd if=/dev/zero of=file bs=1M count=1K && \
mkfs -t ext3 file && \
mkfs /usr/local/cargo/cache && mount file /usr/local/cargo/cache

or with tmpfs like
mount -t tmpfs -o size=2048m tmpfs /usr/local/cargo/registry
But then I need it for buildx builder container. I can't "just mount filesystem" in container during build.
Possible solution can be create and mount new small filesystem for something where placed buildx container — like

dd if=/dev/zero of=/file bs=1M count=6K
mkfs-t ext4 /file
mount /file /var/lib/docker

but again — how can I replace docker work storage on a shared hosting runner?

iav commented

I found.
Something like

  -
        name: create small fs for docker cache
        run: |
          df -h
          sudo swapon --show
          sudo dd if=/dev/zero of=/swapfile1 bs=1M count=6K 
          sudo chmod 600 /swapfile1
          sudo mkswap /swapfile1
          sudo swapon /swapfile1
          sudo swapon --show
          sudo free -h
          sudo systemctl stop docker
          sudo mount -t tmpfs -o size=9G tmpfs /var/lib/docker
          df -h
          sudo systemctl start docker