Upload large files to file storage
bfirsh opened this issue · 6 comments
If you try to pass a large file it fails because the data URL is too large.
replicate.exceptions.ReplicateError: HTTP error: (413, 'Request Entity Too Large')
The solution is to upload it to cloud storage somewhere and pass a URL.
Maybe we could integrate with something like https://upload.io/ . Or if we have an upload API some day, integrate with that.
See also
I ran into this error trying to transcribe some mp4 files. I understand the problem, and I attempted a few URL solutions (as this post suggests), dropbox and google drive. I uploaded my mp4 files to both dropbox and google drive, along with some small wav files to test as well. With all of these URLs, I got the same error:
/tmp/tmpq628bxnt: Invalid data found when processing input
Not sure what URLs you had success with but would love to see if anyone has any examples of solutions to this large file problem.
Hi, @natelalor. Sorry to hear that you're having problems with files uploaded to external CDNs. Can you share the ID of a prediction that failed in this way?
Hi, @natelalor. Sorry to hear that you're having problems with files uploaded to external CDNs. Can you share the ID of a prediction that failed in this way?
Hey @mattt, thanks for the swift reply. How do I share the prediction ID? I just copied a few lines of your code off of your github repo, so obviously very new to replicate.
Also, after some more digging, I realize this may be an ffmpeg error on my end, relating to this post.
@natelalor The easiest way to get the ID of a prediction you made with the API would be to find it on your dashboard on Replicate.com. If you find it there, you can share the URL or the ID with me.
@mattt thank you for the clarification! I just made a new prediction, same error, with the URL: https://replicate.com/p/zglj7pjb5wwd4nof4uura652oa
It is a sample audio which you can find here.
It is a small wav file, which worked for me using the API, but sending it via a dropbox URL made the error. This was for me to test whether it was the URLs I was sending causing the errors, instead of the files I was sending. Thus, it was the URLs that starting kicking errors for me.
Also, I just wanted to add that I uninstalled ffmpeg on my end hoping for something new, but I still get the same error. For clarification if it may help, I am on an M1 Mac using OS Ventura 13.4.1.