Mender Image Upgrade failed - can't update bootloader
Buanderie opened this issue · 7 comments
Same as #190 maybe. Not sure.
I've built an image with mender support for my Jetson Nano EMMC, and can't get to install the mender artifact (performed as a test for future OTA updates).
root@jetcam:~# mender install jetcam-image-minimal-jetson-nano-devkit-emmc.mender
INFO[0000] Loaded configuration file: /var/lib/mender/mender.conf
INFO[0000] Loaded configuration file: /etc/mender/mender.conf
INFO[0000] Mender running on partition: /dev/mmcblk0p1
INFO[0000] Start updating from local image file: [jetcam-image-minimal-jetson-nano-devkit-emmc.mender]
Installing Artifact of size 89011200...
INFO[0000] No public key was provided for authenticating the artifact
INFO[0000] Update Module path "/usr/share/mender/modules/v3" could not be opened (open /usr/share/mender/modules/v3: no such file or directory). Update modules will not be available
INFO[0000] Opening device "/dev/mmcblk0p18" for writing
INFO[0000] Native sector size of block device /dev/mmcblk0p18 is 512 bytes. Mender will write in chunks of 1048576 bytes
.............................................................. - 100 %
INFO[0041] All bytes were successfully written to the new partition
INFO[0041] The optimized block-device writer wrote a total of 4743 frames, where 0 frames did need to be rewritten (i.e., skipped)
INFO[0042] Wrote 4972347392/4972347392 bytes to the inactive partition
INFO[0042] Enabling partition with new image installed to be a boot candidate: 18
INFO[0042] Executing script: ArtifactInstall_Leave_80_bl-update
INFO[0042] Collected output (stderr) while running script /var/lib/mender/scripts/ArtifactInstall_Leave_80_bl-update
ERR: cannot perform bootloader update
---------- end of script output
ERRO[0042] ArtifactInstall_Leave script failed: statescript: error executing 'ArtifactInstall_Leave_80_bl-update': 1 : exit status 1
Rolling back Artifact...
INFO[0042] Rolling back to the active partition: (1).
ERRO[0042] statescript: error executing 'ArtifactInstall_Leave_80_bl-update': 1 : exit status 1
So, I tried to run the ArtifactInstall_Leave_80_bl-update
separately to investigate the matter:
root@jetcam:~# /var/lib/mender/scripts/ArtifactInstall_Leave_80_bl-update
Installing NVIDIA bootloader update payload
mount: /tmp/nvbup.r77OHf: /dev/mmcblk0p1 already mounted on /.
ERR: Missing /opt/ota_package directory in installed rootfs
Weird. Since the /opt/ota_package/bl_update_payload is there :'(
root@jetcam:~# ls /opt/ota_package/bl_update_payload -alh
-rw-r--r-- 1 root root 5.6M Mar 9 2018 /opt/ota_package/bl_update_payload
root@jetcam:~#
Rerunning the script a second time gives:
Installing NVIDIA bootloader update payload
mount: /tmp/nvbup.GpF3jc: /dev/mmcblk0p1 already mounted on /.
ERR: Missing /opt/ota_package directory in installed rootfs
Not sure that this is the actual partition that should be used for BL update
I have no idea what to do next, at this point... I'll keep investigating though.
Any idea ?
Thanks
@Buanderie which branch are you using? I can try to reproduce.
I think this is the actual error
INFO[0042] Executing script: ArtifactInstall_Leave_80_bl-update
INFO[0042] Collected output (stderr) while running script /var/lib/mender/scripts/ArtifactInstall_Leave_80_bl-update
ERR: cannot perform bootloader update
So, I tried to run the ArtifactInstall_Leave_80_bl-update separately to investigate the matter:
mount: /tmp/nvbup.r77OHf: /dev/mmcblk0p1 already mounted on /.
I think this won't work unless it runs as a part of the update - since the chroot at https://github.com/mendersoftware/meta-mender-community/blob/a8247d94d660ea2322293eff44bd9c343a4b05fd/meta-mender-tegra/recipes-mender/tegra-state-scripts/files/redundant-boot-install-script-uboot#L37 is picking the wrong variable for ${new_boot_part}
. You'll need to modify that variable yourself if you want to try running outside mender update.
Weird. Since the /opt/ota_package/bl_update_payload is there :'(
It's running from the chroot, see https://github.com/mendersoftware/meta-mender-community/blob/a8247d94d660ea2322293eff44bd9c343a4b05fd/meta-mender-tegra/recipes-mender/tegra-state-scripts/files/redundant-boot-install-script-uboot#L59, so the error message above is probably related and you aren't getting a root mount with your alternate partition as needed in order to perform the update with your test of the ArtifactInstall_Leave_80_bl-update
script.
@Buanderie which branch are you using? I can try to reproduce.
I'm using the dunfell branch
I think this is the actual error
INFO[0042] Executing script: ArtifactInstall_Leave_80_bl-update INFO[0042] Collected output (stderr) while running script /var/lib/mender/scripts/ArtifactInstall_Leave_80_bl-update ERR: cannot perform bootloader update
So, I tried to run the ArtifactInstall_Leave_80_bl-update separately to investigate the matter:
mount: /tmp/nvbup.r77OHf: /dev/mmcblk0p1 already mounted on /.
I think this won't work unless it runs as a part of the update - since the chroot at https://github.com/mendersoftware/meta-mender-community/blob/a8247d94d660ea2322293eff44bd9c343a4b05fd/meta-mender-tegra/recipes-mender/tegra-state-scripts/files/redundant-boot-install-script-uboot#L37 is picking the wrong variable for
${new_boot_part}
. You'll need to modify that variable yourself if you want to try running outside mender update.Weird. Since the /opt/ota_package/bl_update_payload is there :'(
It's running from the chroot, see https://github.com/mendersoftware/meta-mender-community/blob/a8247d94d660ea2322293eff44bd9c343a4b05fd/meta-mender-tegra/recipes-mender/tegra-state-scripts/files/redundant-boot-install-script-uboot#L59, so the error message above is probably related and you aren't getting a root mount with your alternate partition as needed in order to perform the update with your test of the
ArtifactInstall_Leave_80_bl-update
script.
I can try and produce an artifact with a modified version of the script with some debug output, though
I've verified no issues with jetson-nano-devkit-emmc
on dunfell
with demo-image-base
at d2fe429
root@jetson-nano-devkit-emmc:~# mender install /tmp/demo-image-base-jetson-nano-devkit-emmc.mender
INFO[0000] Loaded configuration file: /var/lib/mender/mender.conf
INFO[0000] Loaded configuration file: /etc/mender/mender.conf
INFO[0000] Mender running on partition: /dev/mmcblk0p1
INFO[0000] Start updating from local image file: [/tmp/demo-image-base-jetson-nano-devkit-emmc.mender]
Installing Artifact of size 166818304...
INFO[0000] No public key was provided for authenticating the artifact
INFO[0000] Update Module path "/usr/share/mender/modules/v3" could not be opened (open /usr/share/mender/modules/v3: no such file or directory). Update modules will not be available
INFO[0000] Opening device "/dev/mmcblk0p18" for writing
INFO[0000] Native sector size of block device /dev/mmcblk0p18 is 512 bytes. Mender will write in chunks of 1048576 bytes
.............................................................. - 100 %
INFO[0030] All bytes were successfully written to the new partition
INFO[0030] The optimized block-device writer wrote a total of 4743 frames, where 0 frames did need to be rewritten (i.e., skipped)
INFO[0031] Wrote 4972347392/4972347392 bytes to the inactive partition
INFO[0031] Enabling partition with new image installed to be a boot candidate: 18
INFO[0031] Executing script: ArtifactInstall_Leave_80_bl-update
Use -commit to update, or -rollback to roll back the update.
At least one payload requested a reboot of the device it updated.
root@jetson-nano-devkit-emmc:~#
You might try
tegra-bootloader-update --dry-run /opt/ota_package/bl_update_payload
and see if it complains about a TNSPEC with your setup similar to #190
Okay let's see...
root@jetcam:~# tegra-bootloader-update --dry-run /opt/ota_package/bl_update_payload
/opt/ota_package/bl_update_payload: bad header version
/opt/ota_package/bl_update_payload: No such file or directory
No idea what that should mean though 😄
Okay let's see...
root@jetcam:~# tegra-bootloader-update --dry-run /opt/ota_package/bl_update_payload /opt/ota_package/bl_update_payload: bad header version /opt/ota_package/bl_update_payload: No such file or directory
No idea what that should mean though smile
Okay... This error message didn't even exist in the source code of the latest version of tegra-boot-tools
. Weird... Until I figured I had an old recipe of tegra-boot-tools
building an old version of it (2.3.0 - which was not compatible with my BUP) in my own layer (which has highest priority).
My bad.
Mender install is working, now :)
Thanks for you help, though !