Unsuccessful dietpi backup
armarbe opened this issue · 18 comments
./shrink-backup.log
2024-10-15 10:56:15 [INFO] - Debugging requested, writing to log file ./shrink-backup.log
2024-10-15 10:56:15 [INFO] - Script started with: ./shrink-backup -a /mnt/sda1/Dietpi_casa.img -l --fix
2024-10-15 10:56:15 [INFO] - Zoom speed NOT requested, setting SLEEPING=sleep 1 | ZOOM=false
2024-10-15 10:56:16 [INFO] - -a selected by user, setting ADDED_SPACE to 0 (non-zero value)
2024-10-15 10:56:16 [DEBUG] - ADDED_SPACE=0
2024-10-15 10:56:16 [INFO] - ext4 root filesystem detected
2024-10-15 10:56:16 [DEBUG] - FSTYPE=ext4
2024-10-15 10:56:16 [DEBUG] - LOCAL_DEV_PTUUID=6a1b63b9-bbea-4097-89b1-d001e65cd172 | LOCAL_DEV_PATH=/dev/mmcblk1
------------------------------------------------------------------------------
2024-10-15 10:56:16 [DEBUG] - IMG_FILE=/mnt/sda1/Dietpi_casa.img
2024-10-15 10:56:16 [DEBUG] - PARTITION_TABLE=gpt
2024-10-15 10:56:16 [DEBUG] - UPDATE=false
2024-10-15 10:56:16 [DEBUG] - AUTORESIZE_RUN=true
2024-10-15 10:56:16 [DEBUG] - PROMPTS=true
2024-10-15 10:56:16 [DEBUG] - EXCLUDE_FILE=false
2024-10-15 10:56:16 [DEBUG] - AUTOEXPAND=true
2024-10-15 10:56:16 [DEBUG] - RSYNC_DELETE=--delete-before
2024-10-15 10:56:16 [DEBUG] - F2FS=false
2024-10-15 10:56:16 [DEBUG] - TTY_AVAILABILITY=/dev/tty
------------------------------------------------------------------------------
2024-10-15 10:56:16 [INFO] - -f NOT selected by user, using default exclude directories
2024-10-15 10:56:16 [INFO] - Running function: make_img
------------------------------------------------------------------------------
2024-10-15 10:56:16 [INFO] - Running function: get_dev_variables
2024-10-15 10:56:16 [INFO] - No boot partition detected
2024-10-15 10:56:16 [DEBUG] - LOCAL_DEV_ROOT_PATH=/dev/mmcblk1p1
2024-10-15 10:56:16 [DEBUG] - LOCAL_ROOT_UUID=6de31054-47ff-41e0-aec5-ca00276550c6 | LOCAL_ROOT_PARTUUID=fed3a6e4-0773-4d4f-a97c-f7ac3f70b95c
2024-10-15 10:56:16 [DEBUG] - LOCAL_ROOT_PARTN=1
2024-10-15 10:56:16 [INFO] - Calculating size for dd to cover bootsector and adding 5MiB (5242880 bytes) to overlap into root (only used in img creation)
2024-10-15 10:56:16 [DEBUG] - LOCAL_ROOT_START=16777216 bytes | LOCAL_BOOTSECTOR=16776704 bytes | LOCAL_DDBOOTSECTOR=43007 4k blocks
2024-10-15 10:56:16 [INFO] - Calculating recommended root size
2024-10-15 10:56:16 [DEBUG] - BLOCKSIZE=4096 bytes
2024-10-15 10:56:16 [INFO] - ext4 filesystem detected, using resize2fs to set recommended root size
2024-10-15 10:56:16 [DEBUG] - LOCAL_AUTORESIZE_MIN=9973776384 bytes
2024-10-15 10:56:16 [INFO] - Setting TOTAL (space needed for files on root) to autocalculated size
2024-10-15 10:56:16 [DEBUG] - TOTAL=9973776384 bytes
2024-10-15 10:56:16 [INFO] - Calculating .img file size by adding LOCAL_BOOTSECTOR to TOTAL
2024-10-15 10:56:16 [DEBUG] - TRUNCATE_TOTAL=9990553088 bytes
2024-10-15 10:56:16 [INFO] - Do you want to continue? [y/n]
2024-10-15 10:56:18 [INFO] - Y or y pressed to confirm
------------------------------------------------------------------------------
2024-10-15 10:56:18 [WARNING] - /mnt/sda1/Dietpi_casa.img ALREADY EXISTS!
2024-10-15 10:56:18 [WARNING] - Do you want to overwrite? [y/n]
2024-10-15 10:56:21 [WARNING] - Overwrite confirmed by user
2024-10-15 10:56:21 [WARNING] - Removing: /mnt/sda1/Dietpi_casa.img
2024-10-15 10:56:23 [INFO] - Using dd to create bootsector
------------------------------------------------------------------------------
2024-10-15 10:56:23 [DEBUG] - Running: dd bs=512 count=43007 if=/dev/mmcblk1 of=/mnt/sda1/Dietpi_casa.img conv=noerror,sync status=progress
43007+0 records in
43007+0 records out
22019584 bytes (22 MB, 21 MiB) copied, 0.50638 s, 43.5 MB/s
------------------------------------------------------------------------------
2024-10-15 10:56:25 [INFO] - Using truncate to resize img file to 9527MiB
2024-10-15 10:56:25 [DEBUG] - Running: truncate --size=9990553088 /mnt/sda1/Dietpi_casa.img
2024-10-15 10:56:26 [DEBUG] - Running function: do_loop
2024-10-15 10:56:26 [DEBUG] - LOOP=/dev/loop0
2024-10-15 10:56:27 [DEBUG] - Running: losetup -P /dev/loop0 /mnt/sda1/Dietpi_casa.img
2024-10-15 10:56:27 [INFO] - Running function: set_img_variables
2024-10-15 10:56:27 [INFO] - No boot partition detected
2024-10-15 10:56:29 [DEBUG] - IMG_DEV_ROOT_PATH=
2024-10-15 10:56:32 [INFO] - sgdisk is available on system
2024-10-15 10:56:33 [INFO] - Using sgdisk to remove root partition
2024-10-15 10:56:33 [DEBUG] - Running: sgdisk /dev/loop0 -d 1
2024-10-15 10:56:36 [INFO] - Using parted to recreate root partition
2024-10-15 10:56:36 [DEBUG] - Running: parted -s -a none /dev/loop0 unit B mkpart primary ext4 16777216 100%
2024-10-15 10:56:37 [INFO] - Formatting filesystem
2024-10-15 10:56:38 [DEBUG] - LABEL= | UUID=6de31054-47ff-41e0-aec5-ca00276550c6
2024-10-15 10:56:38 [INFO] - Using mkfs.ext4 to format root filesystem
------------------------------------------------------------------------------
2024-10-15 10:56:38 [DEBUG] - Running: mkfs.ext4 -U 6de31054-47ff-41e0-aec5-ca00276550c6 -L
mke2fs 1.47.0 (5-Feb-2023)
The file does not exist and no size was specified.
------------------------------------------------------------------------------
2024-10-15 10:56:39 [INFO] - Running function: do_e2fsck
2024-10-15 10:56:40 [DEBUG] - Running: e2fsck -p -f
e2fsck: No such file or directory while trying to open
Possibly non-existent device?
------------------------------------------------------------------------------
2024-10-15 10:56:41 [INFO] - Running function: do_mount
2024-10-15 10:56:42 [INFO] - Creating temp directory
2024-10-15 10:56:42 [DEBUG] - Running: mktemp -d -t backup-XXX
2024-10-15 10:56:42 [DEBUG] - TMP_DIR=/tmp/backup-YBP
2024-10-15 10:56:44 [INFO] - Mounting root partition from loop
2024-10-15 10:56:44 [DEBUG] - Running: mount /tmp/backup-YBP
------------------------------------------------------------------------------
2024-10-15 10:56:44 [ERROR] - ROOT MOUNT FAILED:
mount: /tmp/backup-YBP: wrong fs type, bad option, bad superblock on , missing codepage or helper program, or other error.
dmesg(1) may have more information after failed mount system call.
------------------------------------------------------------------------------
2024-10-15 10:56:44 [ERROR] - Cleanup function called with non zero exit code: exit 1
2024-10-15 10:56:44 [DEBUG] - Removing loop in cleanup function: losetup -d /dev/loop0
2024-10-15 10:56:44 [DEBUG] - Removing temp directory in cleanup function: rm -rf /tmp/backup-YBP
2024-10-15 10:56:44 [INFO] - Elapsed time: 00.29
2024-10-15 10:56:44 [DEBUG] - Exiting script
##############################################################################
To Reproduce
sudo ./shrink-backup -a /mnt/sda1/Dietpi_casa.img -l
Expected behavior
Finishing the backup
Harware (please complete the following information):
- Hardware: Orange Pi 5 (aarch64)
- OS: DietPi v9.7.1
Additional context
The root partition is ext4:
/dev/mmcblk1p1 ext4
Thank you for your help_
Hello and thank you for creating a bug report!
2024-10-15 10:56:27 [INFO] - Running function: set_img_variables
2024-10-15 10:56:27 [INFO] - No boot partition detected
2024-10-15 10:56:29 [DEBUG] - IMG_DEV_ROOT_PATH=
The path is empty, that then leads to other errors further down the line...
This is the same error I have been fighting with in a lot of other reports. The final "solution" was to loop the operation a few times and it worked, but I only implemented that on users WITH a boot partition.
But now it clearly shows it can also happen without it (witch makes perfect sense).
I think it has to do with the system being "too fast" for the network. So I need to make it slow down a little and wait for the network to catch up. (at least so far, all with this error was saving the file over the network)
Out of curiosity, do you save the .img file on a network drive or locally?
As you can see, I have already thought bout it but never had any errors after putting that 2s sleep there without having a boot partition...
Line 614 on main, and 626 on testing:
# I have no idea why, but if I do not put this sleep here, IMG_DEV_ROOT_PATH does not get set
sleep 2
#pause 'Press [Enter] key to continue...'
IMG_DEV_ROOT_PATH=$(lsblk -no path,uuid "$LOOP" | grep "$LOCAL_ROOT_UUID" | awk '{print $1}')
debug 'DEBUG' "IMG_DEV_ROOT_PATH=$IMG_DEV_ROOT_PATH"
Let me do some changes on the testing branch and maybe you can try that version? Because I don't have a dietpi to test on right now so to fix this fast, it would be great if you could help me out here.
I will make another post in this thread when I have made the changes and you can try.
Thank you for such a fast reply. I'm using a USB flash drive. It's USB 3.0, although not so fast (like 40 MB/s write speed).
I will try anything you implement, as I would love to backup my SBC to a img file without having to compress it to .tar.gz
In case it helps, this is the output of fdisk -l without the USB flash drive:
Disk /dev/ram0: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk /dev/mtdblock0: 16 MiB, 16777216 bytes, 32768 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mmcblk1: 29.72 GiB, 31914983424 bytes, 62333952 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 6A1B63B9-BBEA-4097-89B1-D001E65CD172
Device Start End Sectors Size Type
/dev/mmcblk1p1 32768 62333918 62301151 29.7G Linux filesystem
USB.
It is sharing the same io as network (on rpi4 at least, not sure about rpi5), but that is beside the point, just my curiosity. :)
I added the loop for non boot partition systems on the testing branch.
Either just download the file manually and try or if you used git to clone:
cd /path/to/shrink-backup
# update the repo
git pull
# switch to testing branch
git checkout testing
# (you might need to make the script executable again with chmod +x) run the script
sudo shrink-backup yada yada yada
# (optional) go back to main
git checkout main
I would really appreciate it if you could try and see if that works. :)
It's still failing at this point:
## Formatting filesystem...
mke2fs 1.47.0 (5-Feb-2023)
The file does not exist and no size was specified.
## Checking img filesystem...
e2fsck: No such file or directory while trying to open
Possibly non-existent device?
## Creating temp directory...
## Mounting img root partition...
mount: /tmp/backup-Qx0: wrong fs type, bad option, bad superblock on , missing codepage or helper program, or other error.
dmesg(1) may have more information after failed mount system call.
!! ROOT MOUNT FAILED!!!
!! Cleanup function called with non zero exit code, something went wrong!!!
## Exiting and cleaning up...
## Please stand by...
## Done.
## Elapsed time: 00.46
Did the loop run correctly?
It should be noted earlier in the logfile.
Because if the loop is not completing and successfully finding the path after 3 times, it should exit with an error at that point.
takes a second look at the code
Well, it WOULD HAVE IF I DIDN'T FORGET TO IMPLEMENT ALL THE CODE... 🤦
Sorry about that... 😇
Try again and please provide full log from that attempt...
If 3 loops times 5s (15s) is not enough, I have other ideas, but lets do this first..
This is the log output after re-running git pull and then the shrink-backup command:
##############################################################################
2024-10-15 13:51:30 [INFO] - Debugging requested, writing to log file ./shrink-backup.log
2024-10-15 13:51:30 [INFO] - Script started with: ./shrink-backup -a /mnt/sda1/Dietpi_casa.img -ly
2024-10-15 13:51:30 [INFO] - Zoom speed NOT requested, setting SLEEPING=sleep 1 | ZOOM=false
2024-10-15 13:51:31 [INFO] - -a selected by user, setting ADDED_SPACE to 0 (non-zero value)
2024-10-15 13:51:31 [DEBUG] - ADDED_SPACE=0
2024-10-15 13:51:31 [INFO] - ext4 root filesystem detected
2024-10-15 13:51:31 [DEBUG] - FSTYPE=ext4
2024-10-15 13:51:31 [DEBUG] - LOCAL_DEV_PTUUID=6a1b63b9-bbea-4097-89b1-d001e65cd172 | LOCAL_DEV_PATH=/dev/mmcblk1
------------------------------------------------------------------------------
2024-10-15 13:51:31 [DEBUG] - IMG_FILE=/mnt/sda1/Dietpi_casa.img
2024-10-15 13:51:31 [DEBUG] - PARTITION_TABLE=gpt
2024-10-15 13:51:31 [DEBUG] - UPDATE=false
2024-10-15 13:51:31 [DEBUG] - AUTORESIZE_RUN=true
2024-10-15 13:51:31 [DEBUG] - PROMPTS=false
2024-10-15 13:51:31 [DEBUG] - EXCLUDE_FILE=false
2024-10-15 13:51:31 [DEBUG] - AUTOEXPAND=true
2024-10-15 13:51:31 [DEBUG] - RSYNC_DELETE=--delete
2024-10-15 13:51:31 [DEBUG] - F2FS=false
2024-10-15 13:51:31 [DEBUG] - TTY_AVAILABILITY=/dev/tty
------------------------------------------------------------------------------
2024-10-15 13:51:31 [INFO] - -f NOT selected by user, using default exclude directories
2024-10-15 13:51:31 [INFO] - Running function: make_img
------------------------------------------------------------------------------
2024-10-15 13:51:31 [INFO] - Running function: get_dev_variables
2024-10-15 13:51:31 [INFO] - No boot partition detected
2024-10-15 13:51:31 [DEBUG] - LOCAL_DEV_ROOT_PATH=/dev/mmcblk1p1
2024-10-15 13:51:31 [DEBUG] - LOCAL_ROOT_UUID=6de31054-47ff-41e0-aec5-ca00276550c6 | LOCAL_ROOT_PARTUUID=fed3a6e4-0773-4d4f-a97c-f7ac3f70b95c
2024-10-15 13:51:31 [DEBUG] - LOCAL_ROOT_PARTN=1
2024-10-15 13:51:31 [INFO] - Calculating size for dd to cover bootsector and adding 5MiB (5242880 bytes) to overlap into root (only used in img creation)
2024-10-15 13:51:31 [DEBUG] - LOCAL_ROOT_START=16777216 bytes | LOCAL_BOOTSECTOR=16776704 bytes | LOCAL_DDBOOTSECTOR=43007 4k blocks
2024-10-15 13:51:31 [INFO] - Calculating recommended root size
2024-10-15 13:51:31 [DEBUG] - BLOCKSIZE=4096 bytes
2024-10-15 13:51:31 [INFO] - ext4 filesystem detected, using resize2fs to set recommended root size
2024-10-15 13:51:31 [DEBUG] - LOCAL_AUTORESIZE_MIN=9974575104 bytes
2024-10-15 13:51:31 [INFO] - Setting TOTAL (space needed for files on root) to autocalculated size
2024-10-15 13:51:31 [DEBUG] - TOTAL=9974575104 bytes
2024-10-15 13:51:31 [INFO] - Calculating .img file size by adding LOCAL_BOOTSECTOR to TOTAL
2024-10-15 13:51:31 [DEBUG] - TRUNCATE_TOTAL=9991351808 bytes
2024-10-15 13:51:31 [INFO] - -y selected by user. prompts are disabled
2024-10-15 13:51:37 [INFO] - 6 seconds passed, user did not stop operation
------------------------------------------------------------------------------
2024-10-15 13:51:37 [WARNING] - Removing: /mnt/sda1/Dietpi_casa.img
2024-10-15 13:51:39 [INFO] - Using dd to create bootsector
------------------------------------------------------------------------------
2024-10-15 13:51:40 [DEBUG] - Running: dd bs=512 count=43007 if=/dev/mmcblk1 of=/mnt/sda1/Dietpi_casa.img conv=noerror,sync status=progress
43007+0 records in
43007+0 records out
22019584 bytes (22 MB, 21 MiB) copied, 0.489206 s, 45.0 MB/s
------------------------------------------------------------------------------
2024-10-15 13:51:42 [INFO] - Using truncate to resize img file to 9528MiB
2024-10-15 13:51:42 [DEBUG] - Running: truncate --size=9991351808 /mnt/sda1/Dietpi_casa.img
2024-10-15 13:51:43 [INFO] - Running function: do_loop
2024-10-15 13:51:43 [DEBUG] - LOOP=/dev/loop0
2024-10-15 13:51:44 [DEBUG] - Running: losetup -P /dev/loop0 /mnt/sda1/Dietpi_casa.img
2024-10-15 13:51:44 [INFO] - Running function: set_img_variables
2024-10-15 13:51:44 [INFO] - No boot partition detected
2024-10-15 13:51:46 [DEBUG] - IMG_DEV_ROOT_PATH=
2024-10-15 13:51:46 [WARNING] - LOOP paths can not be set, retrying in 5 seconds
2024-10-15 13:51:46 [WARNING] - IMG_DEV_ROOT_PATH=
2024-10-15 13:51:51 [WARNING] - LOOP paths can not be set, retrying in 5 seconds
2024-10-15 13:51:51 [WARNING] - IMG_DEV_ROOT_PATH=
2024-10-15 13:51:56 [WARNING] - LOOP paths can not be set, retrying in 5 seconds
2024-10-15 13:51:56 [WARNING] - IMG_DEV_ROOT_PATH=
2024-10-15 13:52:01 [ERROR] - LOOP PATHS CAN NOT BE SET
2024-10-15 13:52:01 [ERROR] - Cleanup function called with non zero exit code: exit 1
2024-10-15 13:52:01 [DEBUG] - Removing loop in cleanup function: losetup -d /dev/loop0
2024-10-15 13:52:01 [INFO] - Elapsed time: 00.31
2024-10-15 13:52:01 [DEBUG] - Exiting script
##############################################################################
Ok, lets do it the way I did with the other users when having this problem.
It will be way easier for you to do these changes locally than me on the branch.
I left the pause function ready to use right at the code we are using here.
Edit shrink-backup with an editor, for example nano and go to line 627 and remove the # to uncomment the line:
(in nano, ctrl+7 lets you enter line number to jump to)
- #pause 'Press [Enter] key to continue...'
+ pause 'Press [Enter] key to continue...'Save and rerun the script.
When you get to the point where it is paused and asks you to press enter to continue, open another terminal and run the code the script is trying to run and see what we get:
lsblk -no path,uuid /dev/loop0 | grep 6de31054-47ff-41e0-aec5-ca00276550c6 | awk '{print $1}'
If you don't get anything from that, you can check the entire device tree with:
lsblk -f
And provide the output here.
If it however DOES find the correct path in the first command, you can go back to the script and press enter to see if it completes correctly that time.
I've changed the script (it does print Press [Enter] key to continue...) and tested both commands when it's paused. The first one didn't show any output, but the second one is:
NAME FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS
loop0
sda
└─sda1 ext4 1.0 98b008e2-9763-49b0-a489-8812784adeb7 53.5G 0% /mnt/sda1
mtdblock0
mmcblk1
└─mmcblk1p1 ext4 1.0 6de31054-47ff-41e0-aec5-ca00276550c6 19.3G 30% /
Ok, so the loop is failing.
This is not the same thing as on the other users, there is something else going wrong here it seems.
According to your logfile, there is nothing wrong going on while looping the img file so this is very strange.
2024-10-15 13:51:43 [INFO] - Running function: do_loop
2024-10-15 13:51:43 [DEBUG] - LOOP=/dev/loop0
2024-10-15 13:51:44 [DEBUG] - Running: losetup -P /dev/loop0 /mnt/sda1/Dietpi_casa.img
Try this.
Edit the script and copy the pause line you uncommented to line 660 and rerun the script, it will pause before creating the loop so we can try to figure it out.
When paused, in another terminal list the mount where the img file is to make sure THAT is working.
ls -l /mnt/sda1
Then list the first available loop with losetup -f (just to see what it reports suggests)
Then manually try to loop the file, lets use another number for it with:
sudo losetup -P /dev/loop5 /mnt/sda1/Dietpi_casa.img
Then lsblk -f to see if THAT loop is being created.
root@DietPi:~# ls -l /mnt/sda1
total 21520
-rw-r--r-- 1 root root 9991355904 Oct 15 14:09 Dietpi_casa.img
drwx------ 2 root root 16384 Oct 15 09:16 lost+found
root@DietPi:~# losetup -f
/dev/loop1
root@DietPi:~# sudo losetup -P /dev/loop5 /mnt/sda1/Dietpi_casa.img
root@DietPi:~# lsblk -f
NAME FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS
loop0
loop5
sda
└─sda1 ext4 1.0 98b008e2-9763-49b0-a489-8812784adeb7 53.5G 0% /mnt/sda1
mtdblock0
mmcblk1
└─mmcblk1p1 ext4 1.0 6de31054-47ff-41e0-aec5-ca00276550c6 19.3G 30% /
And the log file output is:
2024-10-15 14:57:38 [DEBUG] - IMG_DEV_ROOT_PATH=
2024-10-15 14:57:38 [WARNING] - LOOP paths can not be set, retrying in 5 seconds
2024-10-15 14:57:38 [WARNING] - IMG_DEV_ROOT_PATH=
2024-10-15 14:57:43 [WARNING] - LOOP paths can not be set, retrying in 5 seconds
2024-10-15 14:57:43 [WARNING] - IMG_DEV_ROOT_PATH=
2024-10-15 14:57:48 [WARNING] - LOOP paths can not be set, retrying in 5 seconds
2024-10-15 14:57:48 [WARNING] - IMG_DEV_ROOT_PATH=
2024-10-15 14:57:53 [ERROR] - LOOP PATHS CAN NOT BE SET
2024-10-15 14:57:53 [ERROR] - Cleanup function called with non zero exit code: exit 1
2024-10-15 14:57:53 [DEBUG] - Removing loop in cleanup function: losetup -d /dev/loop0
2024-10-15 14:57:53 [INFO] - Elapsed time: 48.12
2024-10-15 14:57:53 [DEBUG] - Exiting script
##############################################################################
I freed up a rpi4 to install dietpi, and run tests locally, this is very strange to me.
I will come back after running the script locally.
Thank you so much for your efforts ❤
So, I installed the latest version of dietpi (bookworm) in minimal version. Installed required applications like rsync and such.
Mounted a usb stick formatted in exfat and mounted it in /mnt/sda1. I see you have yours in ext4 but that does not matter.
Ran the script without failure, but one thing immediately stood out to me, on rpi there is a boot partition, and the partition table is msdos not gpt, is there really no boot partition on the opi version?
None of this matters though, the scrip can handle it all. What matters is it seems you can't setup a loop, and it works on mine here wo problems.
This leads to the conclusion there has to be something on your side. Encryption you haven't mentioned maybe?
Instead of the -f flag with lsblk, after a "successfull" loop creation, try lsblk -o name,maj:min,size. Does it at least get the size of the file?
This seems to be outside of what the script is doing, if you can't loop an img file, there is not really much I can do for you. :(
Edit
I just dove deep to make sure I am not an ahole here blaming others..
I double checked that the img is being dd:d to the correct size:
Disk /dev/mmcblk1: 29.72 GiB, 31914983424 bytes, 62333952 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 6A1B63B9-BBEA-4097-89B1-D001E65CD172
Device Start End Sectors Size Type
/dev/mmcblk1p1 32768 62333918 62301151 29.7G Linux filesystem
32768 x 512 = 16 777 216
2024-10-15 13:51:31 [INFO] - Calculating size for dd to cover bootsector and adding 5MiB (5242880 bytes) to overlap into root (only used in img creation)
2024-10-15 13:51:31 [DEBUG] - LOCAL_ROOT_START=16777216 bytes | LOCAL_BOOTSECTOR=16776704 bytes | LOCAL_DDBOOTSECTOR=43007 4k blocks
(the 4k is incorrect, this is actually 512byte blocks)
16777216 - (one block) 512 = 16776704 (LOCAL_BOOTSECTOR)
16776704 + (5MiB) 5242880 = 22019584
22019584 / 512 = 43007 (the size that is being dd:d to create the img and partition table)
I have not heard of anyone where this is not enough for the partition table to show and work, the possibility is there, but slim...
I have an idea how you can test this:
Download the latest rpi2-3-4-bookworm and also the latest for your opi5.
Use the --loop function in the script (or loop them manually). Don't forget to remove the loop with sudo losetup -d /dev/loop0 or wherever it got looped to so you don't end up with a bunch of loops everywhere. xD
I just tried with the rpi version here:
$ sudo ./bin/shrink-backup -l --loop /media/fil-fitt/backup/Raspberry\ Pi/rpi4b8gb/DietPi/DietPi_RPi-ARMv8-Bookworm.img
## Debugging requested, writing to log file: ./bin/shrink-backup.log
## Zoom speed NOT requested...
## Looping img file...
## /media/fil-fitt/backup/Raspberry Pi/rpi4b8gb/DietPi/DietPi_RPi-ARMv8-Bookworm.img is looped to /dev/loop0
##############################################################################
# NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
# loop0 7:0 0 851,9M 0 loop
# ├─loop0p1 259:8 0 128M 0 part
# └─loop0p2 259:9 0 722,9M 0 part
##############################################################################
## Done.
$ lsblk
lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
loop0 7:0 0 851,9M 0 loop
├─loop0p1 259:8 0 128M 0 part
└─loop0p2 259:9 0 722,9M 0 part
If you can successfully mount both images and get a proper lsblk output with partitions, there is something wrong with the script and we can go further, but before we absolutely know that, there is no reason to even try.
Edit
Most of this is incorrect, since I later learned the partitions on the installation images are not how the device will be, but I leave the post for history
End edit
I downloaded the latest opi5 image and mounted it and....
sudo ./bin/shrink-backup -l --loop /media/fil-fitt/temp/DietPi_OrangePi5-ARMv8-Bookworm.img
## Debugging requested, writing to log file: ./bin/shrink-backup.log
## Zoom speed NOT requested...
## Looping img file...
## /media/fil-fitt/temp/DietPi_OrangePi5-ARMv8-Bookworm.img is looped to /dev/loop0
##############################################################################
# NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
# loop0 7:0 0 1,1G 0 loop
# ├─loop0p1 259:8 0 1,1G 0 part
# └─loop0p2 259:9 0 1M 0 part
##############################################################################
## Done.
There is a boot partition. Does not necessarily mean you can not set it up to not use one, but you have def done something outside the scope of a "normal setup" it seems.
Again, should not matter because the script does not require a boot partition as I showed with the dd calculations earlier, and it can handle both msdos and gpt partition table, it should not matter...
I checked, and the opi img is indeed a gpt table (compared to rpi that is still old msdos), but why no boot partition on yours...
Well, you have to come back with some info on how it works for you before moving on, because without that information, not much I can do here since I do not own a opi5... :(
Edit
Sorry for spamming, but I really want my script to work as intended.. :)
I found old images and downloaded bullseye for opi5:
$ sudo ./bin/shrink-backup -l --loop /media/fil-fitt/temp/DietPi_OrangePi5-ARMv8-Bullseye.img
## Debugging requested, writing to log file: ./bin/shrink-backup.log
## Zoom speed NOT requested...
## Looping img file...
## /media/fil-fitt/temp/DietPi_OrangePi5-ARMv8-Bullseye.img is looped to /dev/loop1
##############################################################################
# NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
# loop1 7:1 0 1,1G 0 loop
# ├─loop1p1 259:8 0 1,1G 0 part
# └─loop1p2 259:9 0 1M 0 part
##############################################################################
## Done.
You only entered a version in OP, does not tell me much, but bullseye for opi5 also has a boot partition....
Are you sure the partition table you are backing up from is actually ok, and THAT is not the reason the img is not able to be looped?
What is in your /etc/fstab?
Edit 2
I went the extra extra mile to make sure this is not a gpt thing or smthn, by running the dd command on the bullseye img file using the exact calculations used in the script, and after the operations looped the created img file:
<removed in edit>
As you can see, it works and finds the root partition at 1M, exactly how we want it in the script.
(After this the script modifies the file, removes the partition, recreates it etc, so this is exactly how it should look)
But again, this is WITH a boot partition, but IT SHOULD NOT MATTER.
You have to clarify a bit more...
Thank you so much for your efforts. You are doing too much for me :)
I have run the following commands, as advised:
root@DietPi:~# lsblk -o name,maj:min,size
NAME MAJ:MIN SIZE
loop0 7:0 9.3G
sda 8:0 57.7G
└─sda1 8:1 57.7G
mtdblock0 31:0 16M
mmcblk1 179:0 29.7G
└─mmcblk1p1 179:1 29.7G
root@DietPi:~# cat /etc/fstab
# You can use "dietpi-drive_manager" to setup mounts.
# NB: It overwrites and re-creates physical drive mount entries on use.
#----------------------------------------------------------------
# NETWORK
#----------------------------------------------------------------
#----------------------------------------------------------------
# TMPFS
#----------------------------------------------------------------
tmpfs /tmp tmpfs size=3968M,noatime,lazytime,nodev,nosuid,mode=1777
tmpfs /var/log tmpfs size=50M,noatime,lazytime,nodev,nosuid
#----------------------------------------------------------------
# MISC: ecryptfs, vboxsf, glusterfs, mergerfs, bind, Btrfs subvolume
#----------------------------------------------------------------
#----------------------------------------------------------------
# SWAP SPACE
#----------------------------------------------------------------
#----------------------------------------------------------------
# PHYSICAL DRIVES
#----------------------------------------------------------------
UUID=6de31054-47ff-41e0-aec5-ca00276550c6 / ext4 noatime,lazytime,rw 0 1
Further details of Dietpi version:
root@DietPi:~# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
Hope I didn't forget anything.
Hope I didn't forget anything.
- Can you loop other images?
- HOW DID YOU INSTALL?????
- Other opi5 installation instructions seems to indicate you should have a boot partition on opi5, why not yours?
Before I see proof that it is the script doing something wrong, there is nothing I can do.
loop0 7:0 9.3G
I don't know what this loop is, I see nothing on your system that is 9.3G.
Edit
Actually, this could make sense, if this is the size the script calculates as size needed for the img, this could be after the file is resized with truncate.
2024-10-15 13:51:31 [INFO] - ext4 filesystem detected, using resize2fs to set recommended root size
2024-10-15 13:51:31 [DEBUG] - LOCAL_AUTORESIZE_MIN=9974575104 bytes
2024-10-15 13:51:31 [INFO] - Setting TOTAL (space needed for files on root) to autocalculated size
2024-10-15 13:51:31 [DEBUG] - TOTAL=9974575104 bytes
2024-10-15 13:51:31 [INFO] - Calculating .img file size by adding LOCAL_BOOTSECTOR to TOTAL
2024-10-15 13:51:31 [DEBUG] - TRUNCATE_TOTAL=9991351808 bytes
9991351808 / 1024 / 1024 / 1024 = 9,30517148972
9.3G. Ok nvm, it makes sense...
End edit
I am starting to suspect that you have done something like this https://github.com/kaveh-kaviani/Tutorials/blob/main/content/sbc/orange-pi/orange-pi-5/boot-linux-from-emmc/README.md
Ie moved your root to another device but keep your boot somewhere else? For example /dev/mtdblock0
If that is the case, well, this script will not work.
As stated in the readme:
Default device that will be backed up is determined by scanning what disk-device root resides on.
This means that if boot is a partition, that partition must be on the same device as root.
I do not own a opi5 so I can not test further. All I know is the script works on older opi:s without a boot partition (using armbian for example). The spi should not be a problem (other than your img file will be without boot information. that will stay in the spi), what is important is that your root device is not corrupted or strange in any way so a partial dd that the script relies on creates a proper partition table.
But again, the above is about the img being bootablet, it has nothing to do with the script succeeding or not, a boot partition is NOT NEEDED for the script to succeed. What is failing is either the dd or the looping of the img file, you need to prove you CAN loop OTHER img files.
Nothing of this makes sense to me.
You need to disclose EVERYTHING if you want me to be able to help.
Maybe no boot partition IS the standard config, you have to inform me closer because the documentation and tutorials about opi5 and dietpi does not give me the info I need.
I really want this to work, but without information, there simply is nothing I can do. :(
I guess you can increase the size that is being dd:d as a longshot, that would be the only thing I can think of.
On row 387, change:
- LOCAL_DDBOOTSECTOR=$(( (LOCAL_BOOTSECTOR + 5242880) / 512 )) # 512B blocks, 5242880 = 5MiB in bytes
+ LOCAL_DDBOOTSECTOR=$(( (LOCAL_BOOTSECTOR + 10485760) / 512 )) # 512B blocks, 10485760 = 10MiB in bytesThen rerun the script.
It IS very strange that it removes and recreates the partition without trowing errors. That could indicate the device IS there, but the loop is not correct for some reason.
So if you can loop another img file like I suggested in earlier posts (download an img from dietpi for example and use the script with --loop, or manually loop the img), we can see if we can force a reprobe of the loop and maybe it will show so we can set variables correctly...
IF YOU CREATE AN ISSUE, PLEASE MAKE SURE TO FOLLOW THROUGH!!!
There is nothing I can do without further information.
I spent a few hours trying to help out, complete waste of time.