jpmorganchase/jupyter-fs

Quick questions for a download optimizer

JustinMingo opened this issue · 0 comments

JustinMingo commented

I’m prototyping a download optimizer for research. It checks all available filesystems and then tests to copy to the one which is the quickest to upload to.

Quick questions to save a lot of time:

  1. It’s possible to copy files between S3, OSFS and Samba, right?
  2. Is it possible to check for storage space?
  3. How can I reference the opened filesystems to loop through them in Python?

I would then download a testfile to each of them and time it.

Thank you so so much in advance for the pointers!