Upload of large files is never chunked
Closed this issue · 3 comments
Hi,
When uploading large files like 100 Mo or bigger ones (1 To), these are never chunked.
I would expect the client to offer a way to enforce chunk encoding or automatically chunks large files.
An alternative to the current method which tries to always avoid chunk encoding would be to use chunk encoding when a chunk_size is provided (see #167).
What we could also do is add an extra flag to enforce chunks.
Thanks,
Hi @Ph0tonic. Could you please elaborate why you need to chunk? What issues do you run into?
The reason why we don't prefer chunked-encoding is that not a lot of servers support chunked encoding in PUT. This is mostly older servers, but we do bump into them time to time.
See iterative/dvc#4796.
Hi,
So I will have to test it again, but we had some trouble going through a proxy with non-chunked encoding. I understand your point of view and I will check again with our new set-up to see if we still have this issue.
Thanks for your answer
Any updates?