Online show output of backup process
derskythe opened this issue · 5 comments
Hello, I somehow started the backup manually. After a while, I paid attention to the screen and did not know whether it was frozen or not and pressed the "enter" key several times. An hour later, realized that the process looks like frozen and only then realized that my input was a repeated command.
I pressed it only for the purpose of checking if the system hung. If the system displayed messages to me, I would be calm.
My pull request is about the script being written so that it only outputs when the process is finished, which is kinda weird. I added a function and remake the output, and added a few additional messages to show the progress.
As an alternative, perhaps take a look at IOTstackBackup. It's intended to be run from cron rather than from the command line so it always writes its output to a log file (and the log also becomes part of the backup). However, if I want to monitor a backup job while it is running, I can do:
$ tail -f ~/IOTstack/backups/2023-04-07_2338.iot-hub.backup-log.txt
As an alternative, perhaps take a look at IOTstackBackup
Wow, so much bash I never seen before.
I have wrote something to additional backup some directories, but I tired fast with bash and turn to pwsh.
By the way, my static analyzer angry on your -1 exit codes. I think it's byte type or something like C uint8_t, so it can't be less 0.
tail -f
Thank you, I know. But I think I am not alone who wasting couple of hours with this non-responsive script.
My negative exit codes are a very old and very bad habit that I don't seem to be able to get out of. I will put it on my to-do list to fix those. Thanks for mentioning it.
"Fixing" the install stuff (eg IOTstack adopts PiBuilder, perhaps after conversion to Ansible) would be reasonably straightforward - really just needs agreement on approach and scope.
"Fixing" backup/restore is a lot trickier - if you care about forwards/backwards compatibility, which I rate very highly. I think the IOTstack-supplied approach and IOTstackBackup approach will probably need to remain separate.
On the topic of "additional backup some directories", you could take a closer look at snapshot_raspbian_system. The assumption is that IOTstackBackup takes care of everything critical inside of the IOTstack directory, while snapshot_raspbian_system is there to grab other things that might be of interest if you needed to recover a busted system.
I have cron running iotstack_backup twice a day on important Pis, but it only runs snapshot_raspbian_system once a week. I run it by hand if I make any significant changes in, say, /etc but, otherwise, weekly is plenty and it's all there if I ever need to rebuild.
If you build with PiBuilder,it snapshots /etc as /etc-baseline before it does anything. Then, snapshot_raspbian_system always grabs both /etc and /etc-baseline. You can diff those two directories and quickly figure out the answer to "what did I changed during the lifetime of this Pi?" as you set about rebuilding a busted Pi.
Fortunately, my experience with restoring from backup is limited to a couple of times.
For myself, I found a combination of an Ansible script that does the initial installation and configuration of me as a user and my environment and a pwsh script that restores the stack and rewrites my certificates and SSH keys.
During recovery process, I thought about switching to XFS and the second recovery from backup happened for this reason. The system could not start in any way if root partition was converted to XFS. As a result, I have to keep the root partition on the Linux partition, and move /home and /opt to the XFS.
Now I'm starting to think about the BSD on berry, it seems like they brought a docker to them, but at the same time it's not Linux, which is either in a panic or lost partitions.
@Paraphraser do seen developers or someone who just give any reaction to PR/Issue in this repo?