Use "size on disk" instead of "file size" to calculate disk usage
pfrazee opened this issue · 1 comments
Dat makes use of sparse files internally, which causes the "file size" projection to be larger than the actual disk usage is. This causes the size of all history for an archive to be measured, when we may only have partial history on the disk.
https://hashbase.io/dwblair/row-001 is an example of this. It's calculated at 13-14GB, but the on-disk usage is only 4-5GB in reality.
Fixing this will be important for disk-usage management in general. Users may want to delete unwanted history, and without this change that deletion would have no effect on their quota. Fixing this is also important to just be fair to folks.
I uploaded a 1GB archive today and when the upload was paused I exited Beaker and restarted it. I went to hashbase.io and uploaded the same archive (the Add Archive button was grayed out, but when I clicked Browse it went to the upload progress page). After about 630MB was uploaded it paused and I did the same process. Now it shows that the archive is fully uploaded, but I'm not sure if that is true. My usage shows over 2GB, so it is confusing what the actual status is.