fohrloop/dash-uploader

Uploading files to remote servers

yigitsoy opened this issue · 2 comments

yigitsoy commented

Thanks for developing this great tool, which is much more helpful than the vanilla upload of dash.

I am trying to upload files from users's local machines to remote file hosting servers (e.g. s3). I think this would be a great feature to the du where we can bypass any server side processing and directly send the files from the frontend.

How I imagine this is

  • du is configurable so that it uploads to a URL instead of saving it server side
  • Server provides a callback to the client where for each file (ideally relative path to the users local selection root) a PUT URL is generated
  • User selects files (and folders)
  • For each selected file, the client requests the PUT URL from the server and PUTs to the file to the URL

A great example is here: https://dev.to/kitsunekyo/upload-to-aws-s3-directly-from-the-browser-js-aws-sdk-v3-1opk

It would be also great if the server callback can also accepts the type of the file and decides whether to reject it or generate the URL.

Further improvement would be to keep the hierarchy of the selected files while requesting URL, so that the structure is maintained in the remote file hosting servers.

fohrloop commented

Hi @yigitsoy, nice to hear you like the tool!

About uploading directly to a remote file hosting server (e.g. S3):
What you are suggesting sound extremely interesting and also useful for many people![1] Unfortunately I don't believe I will personally have time in near future implementing this, as I have no ongoing work or personal projects that are needing this functionality (or dash-uploader), and my free time is so limited. I'm mainly developing this for fun and as "weekend puzzles" when I don't have higher priority stuff going on.

That being said, I would be really open to PR's trying to address this. If you want to try, the best way would be to continue on the dev branch (currently having version 0.7.0-a1), and read the CONTRIBUTING.md.

Ps. I did not read the mentioned dev.to article all the way, but would that support uploading large files as small chunks? What logic would be used to merge the chunks into a big files at the S3 side?

About server callback accepting the type of the file:
Do you think this is still needed even if the upload component can (and will) filter by file type?[2]

About keeping the hierarchy of the selected files:
Yeah this might or might not be possible. See Issue 76. If it is possible, it will be implemented some day.

[1] It would also help with integration with services like Heroku (Issue 16)
[2] With filetypes argument of the du.Upload.

yigitsoy commented

Hi,

Thanks for the prompt response.

Hi @yigitsoy, nice to hear you like the tool!

About uploading directly to a remote file hosting server (e.g. S3): What you are suggesting sound extremely interesting and also useful for many people![1] Unfortunately I don't believe I will personally have time in near future implementing this, as I have no ongoing work or personal projects that are needing this functionality (or dash-uploader), and my free time is so limited. I'm mainly developing this for fun and as "weekend puzzles" when I don't have higher priority stuff going on.

No worries, I appreciate your contribution already. Looking into your description of how dash-uploader works and potential problems with chunks in the linked issue, this seems (at least to me) harder than what I imagined actually.

That being said, I would be really open to PR's trying to address this. If you want to try, the best way would be to continue on the dev branch (currently having version 0.7.0-a1), and read the CONTRIBUTING.md.

I will try to play around it myself first then, despite lacking frontend experience.

Ps. I did not read the mentioned dev.to article all the way, but would that support uploading large files as small chunks? What logic would be used to merge the chunks into a big files at the S3 side?

I though the aws sdk was handling the streaming of the file (probably in chunks?) to s3 directly and s3 server must be able to combine them. But I assume the sdk is taking the entire file blob and itself sending in chunks.

About server callback accepting the type of the file: Do you think this is still needed even if the upload component can (and will) filter by file type?[2]

Yeah, right, I missed that, that would be sufficient actually.

About keeping the hierarchy of the selected files: Yeah this might or might not be possible. See Issue 76. If it is possible, it will be implemented some day.

Yes, I know, just wanted to put into my wish list :)

[1] It would also help with integration with services like Heroku (Issue 16) [2] With filetypes argument of the du.Upload.

  • 👍2