Pypi package size limit

Question

Pypi package size limit

yifeif opened this issue 8 years ago · 20 comments

Hi,

I'm running into the following issue when trying to twine upload a package around 80MB:
HTTPError: 400 Client Error: distribution file too large for url: https://testpypi.python.org/pypi
What is the size limit for pypi package? Is there a way we can up the limit? Thanks.

Answer 1 · 2016-11-26T17:45:03.000Z

We can increase the limit on a package per package basis. I think the default limit is somewhere around 60MB? You'll have to upload to Warehouse (upload.pypi.org/legacy/ and test.pypi.org/legacy/) instead of to current PyPI but it will still show up on both.

Answer 2 · 2016-11-27T08:59:06.000Z

Thanks @dstufft! Will I still be able to use twine or the upload process is completely different?
I got the following by replacing with the given url,
HTTPError: 404 Client Error: Not Found for url: https://testpypi.python.org/legacy

Answer 3 · 2016-11-28T19:08:58.000Z

The hostname in your error message is testpypi.python.org, but Donald’s message talks about test.pypi.org :)

Answer 4 · 2016-11-28T21:45:01.000Z

Our project has been waiting for this to be active but I hadn't realized this was already possible. How do we request a size increase for our project?

Answer 5 · 2016-11-29T03:10:45.000Z

@yifeif You can still use twine, but you need to use either https://test.pypi.org/legacy/ or upload.pypi.org/legacy/ for TestPyPI or PyPI (note the domain and trailing slash). You would also need to tell me the name of your project and why you need it increased (e.g. large image files, lots of compiled .so's, whatever) if you want the file size limit increased for that project.

@matham You can just ask me for one in this issue or via email or whatever. I just need to know the name of the project and why you need it increased (e.g. large image files, lots of compiled .so's, whatever).

Answer 6 · 2016-11-29T21:48:18.000Z

@dstufft that's great!

The project name is kivy.deps.gstreamer https://pypi.python.org/pypi/kivy.deps.gstreamer. It's a dependency of the kivy project and is used by the kivy project for video. The reason for the large size is that it includes all the gstreamer binaries and plugin dlls.

Currently we host it on our site and it's about 120MB. However,could you increase the limit to around 150MB in case the gstreamer distribution gets larger?

Thanks!

Answer 7 · 2016-11-29T22:15:19.000Z

Thanks @merwok and @dstufft! I was totally using the wrong sites :)

May I get a file size limit increased to 100MB for project 'tensorflow-gpu' https://pypi.python.org/pypi/tensorflow-gpu? It is currently at around 85MB.
In additional to the normal tensorflow package, it has much bigger .so for all the GPU supports. Thanks!

Answer 8 · 2016-12-05T18:15:27.000Z

@dstufft gentle ping on the file size increase request. Much appreciated!

Answer 9 · 2016-12-05T18:27:12.000Z

Okay, I've set kivy.deps.gstreamer to 150MB and tensorflow-gpu to 100MB. Note: This will only take affect when uploading to the new site at https://upload.pypi.org/legacy/, but the change will be reflected in both the old and the new site.

Answer 10 · 2016-12-05T18:47:01.000Z

Was able to upload to https://upload.pypi.org/legacy/ just now. Thanks so much @dstufft!

One more request, can we have the same size increase at https://test.pypi.org/legacy/ as well?

Answer 11 · 2016-12-05T18:48:22.000Z

@yifeif done!

Answer 12 · 2016-12-05T18:52:48.000Z

Awesome. Thanks again for all your help @dstufft :)

Answer 13 · 2016-12-05T20:43:56.000Z

@dstufft Thanks a lot!

Answer 14 · 2017-01-18T01:49:24.000Z

Hi @dstufft, we would like to expand tensorflow-gpu's support for more CUDA compute capabilities as mentioned in this thread, which will cause our binary size to go up. Would it be possible to increase tensorflow-gpu size limit to 200MB?
Sorry need to dig up this old thread and really appreciate the help!

Answer 15 · 2017-11-17T11:51:06.000Z

Hi @dstufft,

We're trying to publish project deepspeech-gpu and are running into the same problem as tensorflow-gpu, as we also include a bunch of compiled libraries for CUDA support. Could you bump our limit to 200MB on upload.pypi.org and test.pypi.org?

Thanks!

Answer 16 · 2017-11-17T16:49:43.000Z

New requests for different packages would need their own tickets.

Answer 17 · 2017-11-17T16:53:04.000Z

Thanks. I filed #110

Answer 18 · 2019-06-26T03:14:37.000Z

@dstufft! My Program is advanced tensorflow for edge computing, which is named as tensorflow-tflex, its size is 69.1M, could you increase the upload size limit? Thank you very much!
HTTPError: 400 Client Error: File too large. Limit for project 'tensorflow-tflex' is 60 MB. See https://test.pypi.org/help/#file-size-limit for url: https://test.pypi.org/legacy/

Package site: https://pypi.org/project/tensorflow-tflex/#files

Answer 19 · 2019-08-09T03:31:07.000Z

@dstufft My project is tr (https://pypi.org/project/tr/)
it is a deep learning based OCR system which include many .so files and big deep learning model parameter files(text line detect and text recognition), I want a size around 300Mb to store so many files, Thanks

Answer 20 · 2019-08-09T04:52:18.000Z

Please open new issues at https://github.com/pypa/warehouse/issues for new limit requests.