Inform user why unmounting partition could take a long time or perform separate sync operation before unmount
tgharib opened this issue · 13 comments
I was stuck at "Unmounting partition" for several minutes. I was confused why this was the case because an unmount should take less than a second. I was getting ready to kill the process when I remembered that the kernel buffers data to-be-written-to-disk in RAM. So I performed a sync command, and lo behold, there was still buffered data in RAM.
I recommend either informing the user why an unmount can take several minutes (because data is still being written to USB drive) or to do a separate sync operation before unmount.
To elaborate, this comment explains how to check progress as the data flushes out from RAM.
watch grep -e Dirty: -e Writeback: /proc/meminfo is ran. Dirty and Writeback decrease until they reach roughly 0 (process is complete then).
Still shows: "== Unmounting partition ==" for long time.
Using 0.1.5-2
As @tgharib commented, I could see data flushing from RAM.
Still an issue as today
@eathtespagheti do you use appimage build?
same me, I'm using https://aur.archlinux.org/windows2usb.git.
It took ~10min on this step
This is a known [1, 2] Linux kernel issue, or, more precisely, a configuration issue: by default writeback buffer is very large, tuned for server workloads. It is not limited to windows2usb, you'll see this behavior on a simple big file copy with a file manager.
This could be fixed by either of:
- Reducing writeback buffer. Very simple one time configuration:
echo 'vm.dirty_bytes = 67108864' | sudo tee /etc/sysctl.d/60-dirty.conf
echo 'vm.dirty_background_bytes = 16777216' | sudo tee -a /etc/sysctl.d/60-dirty.conf
sudo sysctl --system
- Enabling Writeback Throttling feature. If you have this issue, most probably your distro compiles kernel without
BLK_WBT_SQflag and you're using default I/O scheduler. If the kernel is compiled withBLK_WBT_MQ(MQ, not SQ), then you can switch the disk tomq-deadlinescheduler to make throttling work. Something like this:
echo mq-deadline | sudo tee /sys/block/sda/queue/scheduler
This could be configured on boot withelevator=mq-deadlinekernel command line.
Appimage build contains autofsync library, which limits writeback data in userspace. It should help and works for me. If it doesn't, please report. ArchLinux AUR script may include autofsync as well (replace 7z with a shell script which sets LD_PRELOAD to autofsync and executes real 7z binary with it).
I don't want windows2usb to tune writeback values or change I/O scheduler automatically, because that's potentially unwanted feature. I totally understand your frustration with Linux defaults, but it has nothing with this tool.
@eathtespagheti do you use appimage build?
I'm using AUR version too, and thanks for the explanation on the "issue"!
I don't want windows2usb to tune writeback values or change I/O scheduler automatically, because that's potentially unwanted feature. I totally understand your frustration with Linux defaults, but it has nothing with this tool.
It is understandable you do not want to mess with linux defaults.
At the same time I landed here by searching for "windows2usb unmounting partition" just today.
In principle, I do not mind waiting for 5-10 minutes for the unmount, but I do mind programs looking unresponsive/stuck.
You can solve this by a simple echo under == Unmounting partition ==:
Note: Unmounting might take up to 10 minutes.
More info: https://github.com/ValdikSS/windows2usb/issues/3
Or something along those lines.
IMHO it is much better having 2 lines explaining what is happening than assuming a linux user wouldn't suspect that the program hang when the simple unmount takes 10 minutes.
Thank you!
Done, the note added.
I'm using https://aur.archlinux.org/windows2usb.git.
It took ~10min on this step aswell,
Note: Unmounting might take up to 10 minutes.
during this step - the "Dirty" memory didn't get written to disk (watch grep -e Dirty: -e Writeback: /proc/meminfo_
I opened up a secondary process, and ran umount on the device (again) and this seemed to cause the cache to start writing to the disk. this would definitely need more testing, but a sleep 5 && umount /dev/sdX (on a secondary thread) might be whats needed if its refusing to drop the cache.
Maybe it would be nice to define "very long".
I saw the message
NOTE: If this process takes very long to complete, your system is misconfigured! More info: https://github.com/ValdikSS/windows2usb/issues/3#issuecomment-771534058
and came here thinking i would need to fix something, but it ended up terminating with success after maybe 3 minutes.
No problem here, v0.2.4 works great! ))