OE4T/tegra-demo-distro

unable to extract nvidia-l4t-graphics-demos_32.5.0 with workaround

supershabam opened this issue · 3 comments

supershabam commented

branches tested:

  • https://github.com/OE4T/tegra-demo-distro/commit/9089b6b920ef21a47b96049e86f8d673490f14ca
  • https://github.com/OE4T/tegra-demo-distro/commit/a008bc408a651c4dfd17f5575cf5a6a4af446e48

build configuration

Build Configuration:
BB_VERSION           = "1.49.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "aarch64-oe4t-linux"
MACHINE              = "jetson-tx2-devkit"
DISTRO               = "tegrademo-mender"
DISTRO_VERSION       = "3.2+snapshot"
TUNE_FEATURES        = "aarch64 armv8a crc"
TARGET_FPU           = ""
meta                 = "HEAD:2ffbb020fe1d9f332fdc58cd93d7a41885850771"
meta-tegra           
contrib              = "HEAD:33e503050fdf69c5e25fcc7d1e9d114a138bc14a"
meta-oe              
meta-python          
meta-networking      
meta-filesystems     = "HEAD:326d19c39389f977d22a745ca3fcf6ab58c235ea"
meta-virtualization  = "HEAD:e7a0ea3e0fc43f6c2ec8fee70ec1315014ffbec1"
meta-mender-core     = "HEAD:f5864bfb32906b87cc24d5b84e74805994f0ef3e"
meta-mender-tegra    = "HEAD:d59118bcdb701aa84c3ecf458f0c4c4c6cd3e6ea"
meta-tegra-support   
meta-demo-ci         
meta-tegrademo       = "master:a008bc408a651c4dfd17f5575cf5a6a4af446e48"

host machine:

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.5 LTS
Release:        18.04
Codename:       bionic

uname -a
Linux buildybarge 5.4.0-65-generic #73~18.04.1-Ubuntu SMP Tue Jan 19 09:02:24 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

steps:

  1. checkout tegra-demo-distro to desired commit
  2. update submodules with git submodule update --init
  3. run setup-env for tx2 and tegrademo-mender . ./setup-env --machine jetson-tx2-devkit --distro tegrademo-mender
  4. (note nvidia/sdkm_downloads is populated in $HOME/Downloads)
  5. execute bitbake for full image bitbake demo-image-full

Before long (after many modules have downloaded but before most start compiling) I run into the following error

DEBUG: Executing python function extend_recipe_sysroot
NOTE: Direct dependencies are ['virtual:native:/home/ubuntu/tegra-demo-distro/layers/meta-oe/recipes-extended/zstd/zstd_1.4.5.bb:do_populate_sysroot', 'virtual:native:/home/ubuntu/tegra-demo-distro/layers/meta/recipes-extended/xz/xz_5.2.5.bb:do_populate_sysroot']
NOTE: Installed into sysroot: ['zstd-native', 'xz-native', 'quilt-native', 'gnu-config-native', 'automake-native', 'gettext-minimal-native', 'autoconf-native', 'libtool-native', 'texinfo-dummy-native', 'm4-native']
NOTE: Skipping as already exists in sysroot: []
DEBUG: sed -e 's:^[^/]*/:/home/ubuntu/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/l4t-nvidia-glheaders/32.5.0-r0/recipe-sysroot-native/:g' /home/ubuntu/tegra-demo-distro/build/tmp/sysroots-components/x86_64/quilt-native/fixmepath /home/ubuntu/tegra-demo-distro/build/tmp/sysroots-components/x86_64/gnu-config-native/fixmepath /home/ubuntu/tegra-demo-distro/build/tmp/sysroots-components/x86_64/automake-native/fixmepath /home/ubuntu/tegra-demo-distro/build/tmp/sysroots-components/x86_64/autoconf-native/fixmepath /home/ubuntu/tegra-demo-distro/build/tmp/sysroots-components/x86_64/libtool-native/fixmepath | xargs sed -i -e 's:FIXMESTAGINGDIRTARGET:/home/ubuntu/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/l4t-nvidia-glheaders/32.5.0-r0/recipe-sysroot:g; s:FIXMESTAGINGDIRHOST:/home/ubuntu/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/l4t-nvidia-glheaders/32.5.0-r0/recipe-sysroot-native:g' -e 's:FIXME_PSEUDO_SYSROOT:/home/ubuntu/tegra-demo-distro/build/tmp/sysroots-components/x86_64/pseudo-native:g' -e 's:FIXME_HOSTTOOLS_DIR:/home/ubuntu/tegra-demo-distro/build/tmp/hosttools:g' -e 's:FIXME_PKGDATA_DIR:/home/ubuntu/tegra-demo-distro/build/tmp/pkgdata/jetson-tx2-devkit:g' -e 's:FIXME_PSEUDO_LOCALSTATEDIR:/home/ubuntu/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/l4t-nvidia-glheaders/32.5.0-r0/pseudo/:g' -e 's:FIXME_LOGFIFO:/home/ubuntu/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/l4t-nvidia-glheaders/32.5.0-r0/temp/fifo.7896:g'
DEBUG: Python function extend_recipe_sysroot finished
DEBUG: Executing python function do_unpack
DEBUG: Executing python function base_do_unpack
NOTE: Unpacking /home/ubuntu/tegra-demo-distro/build/downloads/nvidia-l4t-graphics-demos_32.5.0-20210115151051_arm64.deb to /home/ubuntu/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/l4t-nvidia-glheaders/32.5.0-r0/l4t-graphics-demos/
tar: This does not look like a tar archive
tar: Skipping to next header
tar: Exiting with failure status due to previous errors
ERROR: Unpack failure for URL: 'https://repo.download.nvidia.com/jetson/t186/pool/main/n/nvidia-l4t-graphics-demos/nvidia-l4t-graphics-demos_32.5.0-20210115151051_arm64.deb;subdir=l4t-graphics-demos'. Unpack command PATH="/home/ubuntu/tegra-demo-distro/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/ubuntu/tegra-demo-distro/layers/scripts:/home/ubuntu/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/l4t-nvidia-glheaders/32.5.0-r0/recipe-sysroot-native/usr/bin/aarch64-oe4t-linux:/home/ubuntu/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/l4t-nvidia-glheaders/32.5.0-r0/recipe-sysroot/usr/bin/crossscripts:/home/ubuntu/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/l4t-nvidia-glheaders/32.5.0-r0/recipe-sysroot-native/usr/sbin:/home/ubuntu/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/l4t-nvidia-glheaders/32.5.0-r0/recipe-sysroot-native/usr/bin:/home/ubuntu/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/l4t-nvidia-glheaders/32.5.0-r0/recipe-sysroot-native/sbin:/home/ubuntu/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/l4t-nvidia-glheaders/32.5.0-r0/recipe-sysroot-native/bin:/home/ubuntu/tegra-demo-distro/repos/poky/bitbake/bin:/home/ubuntu/tegra-demo-distro/build/tmp/hosttools" ar x /home/ubuntu/tegra-demo-distro/build/downloads/nvidia-l4t-graphics-demos_32.5.0-20210115151051_arm64.deb data.tar.zst && tar --no-same-owner -xpf data.tar.zst && rm data.tar.zst failed with return value 2
DEBUG: Python function base_do_unpack finished
DEBUG: Python function do_unpack finished

The core problem appears to be the version of tar that is available on ubuntu 18.04 and the usage of zstd compressed data.

The version of tar that I am running is v1.29

tar --version
tar (GNU tar) 1.29

It appears that zstd compression was added to tar at version 1.31. Ubuntu 18.04 doesn't appear to provide tar at a version higher than v1.29 through apt.

workaround

I've downloaded and compiled tar v1.31 from https://www.gnu.org/software/tar/ and am now able to make it past this error.

note: v1.33 of tar introduces a new error with perform_packagecopy for several recipes which is a core helper from the poky meta library

The error from v1.33 tar looks like the following

Exception: subprocess.CalledProcessError: Command 'tar -cf - -C /home/ubuntu/tegra-demo-distro/build-32-4-3/tmp/work/all-oe4t-linux/tzd
ata/2021a-r0/image -p -S . | tar -xf - -C /home/ubuntu/tegra-demo-distro/build-32-4-3/tmp/work/all-oe4t-linux/tzdata/2021a-r0/package' 
returned non-zero exit status 2.

Subprocess output:
tar: ./etc/localtime: Cannot change mode to rwxrwxrwx: Operation not permitted
tar: Exiting with failure status due to previous errors
madisongh commented

Yes, NVIDIA started using XZ-compressed tarballs in at least some of the .deb packages with R32.5.0/JP4.5. I added the xz-native dependency, but was testing builds on Ubuntu 20.04 and later, so I didn't run into the tar compatibility issue.

I think I know how to fix this, but the fix will be over in meta-tegra, rather than here in the distro.

madisongh commented

It took a couple of tries, but I've implemented a fix/workaround for this in meta-tegra and have updated both master and dunfell-l4t-r32.5.0 to include it.

supershabam commented

@madisongh 👏 wow, thanks for the quick work