How can you find the Tus server configuration?
johnloringpollard opened this issue · 3 comments
Not sure this is the best place to ask but I am implementing this and I wanted to test the configuration to find the fastest possible upload speeds. I'm not sure if concurrency is 'on' even though I set it in the Tus::Storage::S3.new options. Also is there a recommendation for chunk size?
Tus::Server.opts[:storage] = Tus::Storage::S3.new(
bucket: "....",
access_key_id: .....,
secret_access_key: .....,
region: "us-east-1",
use_accelerate_endpoint: true,
logger: Logger.new(STDOUT),
retry_limit: 5,
http_open_timeout: 10,
concurrency: { concatenation: 20 }
)
Tus::Server.opts[:redirect_download] = true
I'm not sure what's your specific question, but the best tus server configuration is going to depend on your application and infrastructure.
I'm not sure if concurrency is 'on' even though I set it in the Tus::Storage::S3.new options
None of the tus storages upload chunks concurrently, because it will generally always take slower for data to be transferred from client to server, than from server to S3. The :concurrency
option was for something else, and it has been removed on master
.
Also is there a recommendation for chunk size?
S3 requires minimum of 5MB, but beyond that it depends. If I'm using Falcon, which supports actual streaming, I would not set any chunk size, as setting a chunk size means having to send a new request after each chunk. For other web servers like Puma and Unicorn I'd set 5MB, as currently any chunk that fails needs to be retried from the beginning (I want to improve this in future versions, like tusd
does).
Awesome thank you for the info! I'm still learning about all this resumable stuff.
Have you had anyone use this to integrate with ActiveStorage? I noticed that you can't upload the same file twice because you will run into an ActiveStorage Blobs unique key index issue. I think Tus has to be this way to generate the same ID to resume? I also had to manually run the checksum and submit all details to create the blob and attachment after the file is uploaded.
Thanks for the GEM, it has saved me a lot of time!!
I haven't used it with Active Storage, I'm not sure what it would take.
I'm very glad to hear that it was helpful!