sigoden/dufs

Skip existing files on upload?

Closed this issue · 2 comments

Specific Demand

If I upload a folder with the same name say "Backup", can we skip the files that already exist? rather than try & overwriting them?

We will not support this feature.

We cannot skip files based solely on their existence. What if the content is different, but the file name remains the same?

A better approach might be skipping uploads if a file with the same hash already exists.

However, this also won't work. Calculating the hash requires loading entire files into memory, while the current upload process is streamed and uses less memory. This could overload the browser and slow down the upload.

Cool, thanks for explaining the nuances. I really appreciate dufs btw, it performs quite well so I understand why you'd want to keep that.

I just tried building dufs from source so if I ever really need this feature I can add it myself. It failed at the link step though:

   Compiling dufs v0.41.0 (/ssd/dufferzafar/apps/_rust/dufs)                                                                            
error: linking with `cc` failed: exit status: 1
 
  = note: /home/code/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /sp
home/code/bin/../x86_64-conda-linux-gnu/sysroot/usr/lib/../lib/Scrt1.o: in function `_start':
          (.text+0x12): undefined reference to `__libc_csu_fini'
          /home/code/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: (.t
ext+0x19): undefined reference to `__libc_csu_init'
          collect2: error: ld returned 1 exit status

  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/referen
ce/build-scripts.html#rustc-link-lib)

This is an older ubuntu, so I'll first try it on some other box!