fallocate freeze with large files
camtastic2912 opened this issue · 8 comments
As of a few days ago, transfers of large files using pget result in a freeze of up to two minutes before the transfer starts again.
Setting 'set file:use-fallocate false' in lftp.conf appears to resolve the issue, but I can't remember pre-allocation ever taking this long previously and it's driving me crazy.
Thanks!
I just updated to cygwin 3.1.6 and i don't have this issues.
Primarily this setting should be controlled here
https://github.com/userdocs/LFTP4WIN/blob/master/LFTP4WIN-installer.cmd#L322
Setting sparse file support when generating the system/etc/fstab file.
Please make sure you are updated and this setting is present.
Reverted to 3.1.6 and issue still persists when removing 'set file:use-fallocate false' from the lftp.conf file.
Checked 3.1.5 and same issue.
fstab is configured as "none /cygdrive cygdrive binary,noacl,posix=0,sparse,user 0 0"
It's worth mentioning as well that I've installed Ubuntu 20.04 Linux Subsystem for Windows and tried the same LFTP transfer and this also hangs when pre-allocating disk space.
My mistake, i updated from 3.1.6 to to 3.1.7 - cygcheck (cygwin) 3.1.7
I am not really sure how to help with this issue. Perhaps you have some other program or issue interfering with how your drive or space is managed.
Same issue on 3.1.7 unfortunately.
I did perform a clean install of Windows yesterday and set up LFTP4WIN before anything else to test and the issue was present.
Interesting enough, it may be related to Windows 10 build 2004, which I've only been running for about a week.
All good! I might just need to live with the mystery.
I also have build 2004.
If this problem is a recent one there will be another cause. Perhaps a driver issue or something. I ma not sure how to test or debug cygwin for this particular issue tbh.
Just setup from scratch on an entirely different PC with different hardware and drivers and same issue.
Is some delay when pre-allocating to disk expected or should it not be noticeable at all?
Considering the same issue occurs on LFTP running on Linux Subsystem for Windows I'm hesitant to believe it's an issue with Cygwin.
Either way I'll just leave fallocate disabled.
Thanks for all your help. There's nothing else quite as fit for purpose as LFTP4WIN so just very glad you're still supporting it!
Ok, so looking at it from my experience, i have always had an ssd as my main drive, but experience a similar outcome on a secondary mechanical 2.5 inch drive, With both drives using NTFS.
Using this example: Downloading a 20GB MKV using the pget-to-local command with default settings on Windows 10 2004 using a default install (as of today) of LFTP4WIN. Destination drive either the main sdd or the secondary mechanical.
cygwin sparse files OFF = Just after the pget terminal window opens there will be a freeze while the file space is allocated. It depends on the file size. It can be a few seconds to a minutes.
cygwin sparse files ON = Just after the pget terminal window opens the file starts downloading immediately with no delay or freeze.
The freeze is not a bug but the expected outcome due to the way file space is allocated in Windows vs unix system. The Cygwin sparse file is a workaround (i think) with some performance sacrifice.
From here - https://cygwin.com/cygwin-ug-net/using.html
sparse - Switch on support for sparse files. This option only makes
sense on NTFS and then only if you really need sparse files.
Cygwin does not try to create sparse files by default for
performance reasons.
Conclusion: There is another cause to this problem.
Excellent information. Thanks mate!