Azure/iot-hub-device-update

Yocto Rpi3 Device is restarting again and again in a loop

Closed this issue · 2 comments

manojrampur commented

Hi Team,

I have built a Yocto Kirkstone Image using IotEdge Runtime and the device update-agent, I can see that IoTEdge runtime and device update-agent is active and running.
Created the .swu file for .txt files using cpio

Expected Behavior

When we push the update through the Azure portal, it should install the .swu file in the device and report the status as succeeded in the Azure portal after successful installation.

Current Behavior

When I push the update through the Azure portal, I can see the .swu file is getting installed but the device is rebooting again and again in a loop, and In the Azure portal still the status is In Progress.

Also, I am not able to see the IoTHub Device Update Module. Is it expected when we provision through the Yocto Image?

Device Information

  • Host OS [e.g. Ubuntu 18.04, Windows Server IoT 2019]: Yocto Kirkstone Linux OS
  • Architecture [e.g. amd64, arm32, arm64]: arm32
  • Provisioning Method: module provisioning with azure identity service
    *DU Agent Version: 1.0.1

[Logs]

adu-shell.log
du-agent.log
swupdate.log
swupdate-v2-handler.log
device twin.txt

Please help me to resolve this issue asap.

Thanks in Advance!!

jw-msft commented

From the logs provided, I see that Apply succeeded and required reboot (result code 705 is ADUC_Result_Apply_RequiredImmediateReboot) and the swupdate.log suggests that the new image was written to the raw partition successfully

The likely issue for it doing it over again is that the "installedcriteria" was not updated to that in the update manifest (derived from the import manifest handler properties under the microsoft/swupdate:2 handler section.

Is the https://github.com/Azure/iot-hub-device-update/blob/main/src/extensions/step_handlers/swupdate_handler_v2/tests/testdata/adu-yocto-ab-rootfs-update/payloads/example-a-b-update.sh#L475 example script is being used, perhaps unmodified?

I see a warning message in provided adu-shell.log:

2023-12-01T10:25:04.3151Z 839[839] [I] #  ^[[1;33mWarning:^[[0m ************************************************* [ShowChildProcessLogs]
2023-12-01T10:25:04.3151Z 839[839] [I] #  ^[[1;33mWarning:^[[0m *                    WARNING                    * [ShowChildProcessLogs]
2023-12-01T10:25:04.3152Z 839[839] [I] #  ^[[1;33mWarning:^[[0m *                                               * [ShowChildProcessLogs]
2023-12-01T10:25:04.3152Z 839[839] [I] #  ^[[1;33mWarning:^[[0m * THIS FILE IS FOR DEMONSTRATION PURPOSES ONLY. * [ShowChildProcessLogs]
2023-12-01T10:25:04.3152Z 839[839] [I] #  ^[[1;33mWarning:^[[0m * DO NOT USE THIS FOR YOUR REAL PRODUCT UPDATE! * [ShowChildProcessLogs]
2023-12-01T10:25:04.3153Z 839[839] [I] #  ^[[1;33mWarning:^[[0m *                                               * [ShowChildProcessLogs]
2023-12-01T10:25:04.3153Z 839[839] [I] #  ^[[1;33mWarning:^[[0m *************************************************

If that is the case, then the IsInstalled() function in the script greps /etc/adu-version file for the installed-criteria from the parsed cmd-line args of the script invocation in the as can be seen here:

iot-hub-device-update/src/extensions/step_handlers/swupdate_handler_v2/tests/testdata/adu-yocto-ab-rootfs-update/payloads/example-a-b-update.sh

Line 475 in f9b180d

grep "^$1$" "$software_version_file"

If /etc/adu-version does not contain a line that grep will match with installed-criteria from the manifest's "installedcriteria" handler property (under "handlerProperties"), then it will keep re-installing because it thinks that it's not installed yet.

If /etc/adu-version is part of the new image that was written, then you have to ensure that /etc/adu-version in the new update image has a line with the "installedcriteria" value from the manifest's "handlerProperties".

If /etc/adu-version is NOT part of the new image and, say, /etc/ is in a separate data partition that is left untouched by the A/B update, then you would need to modify the script and/or determine a way to know that the IsInstalled() evaluation is happening after 1st reboot after swupdate was written (e.g. can use fwsetenv to set a uboot variable or touch a file such as /etc/1streboot_inprogress).

The example in its current state cannot not be used in production--the installedcriteria will keep changing with each update and each image will need to have the new installedcriteria encoded inside of it somehow or the swupdate-v2 script will have to know how to determine if the installedcriteria matches.

Summary

Problem:

  • Device keeps re-installing the same update.

Reason

  • The IsInstalled() function in the update script is using a file (/etc/adu-version) to determine if the update is already installed.
  • However, the new update image does not contain this file with the updated installedcriteria value.

Possible solutions:

  • Update the new image: Ensure the new image includes /etc/adu-version with the correct installedcriteria value.
  • Modify the script: Use a different mechanism to store the installedcriteria value (e.g., uboot env var, file in a data partition). Modify IsInstalled() to read the new storage location and verify the update integrity.

Key points:

  • The provided example script is for demonstration only and not suitable for production.

  • The installedcriteria value will change with each update, so it needs to be stored in the image or through another mechanism.

  • The IsInstalled() function should also perform integrity checks on the written image.

Additional information:

  • The warning message in the log indicates that the example script should not be used in production.
  • The script assumes /etc/adu-version is part of the new image, which may not be the case.

Further action:

  • Check if /etc/adu-version is in the new image and contains the correct installedcriteria value.
  • If not, choose and implement one of the suggested solutions.
  • Modify the IsInstalled() function to use the new storage location and perform integrity checks.
jw-msft commented

@manojrampur, From your device twin, it looks like the argument for passing in the installed criteria has a typo:

It has --intalled-criteria 1.0 but it should be --installed-criteria 1.0.
NOTE: the missing "s"

Also, make sure the /etc/adu-version in the .swu image has a line with 1.0

"instructions":{"steps":[{"handler":"microsoft/swupdate:2","files":["f9938843efef12267","fe02407a96dfbe7cf"],"handlerProperties":{"scriptFileName":"example-a-b-update.sh","arguments":"--action-install --intalled-criteria 1.0 --swu-file audio-analytics-update_1.2.swu --work-folder ","installedCriteria":"1.0",