Http.request reads whole body to memory
Opened this issue · 1 comments
GoogleCodeExporter commented
I want to use httplib2 to fetch large files (say, 1G) and there doesn't seem to
be a way to read response body iteratively - the request() method returns whole
body as an in-memory string.
I look at the code and the internal APIs do support iterative reading, but
somewhere later content.read() is called and whole thing is pulled to memory.
Maybe you could expose the internal API via a new method or a flag to request()
(like "read_full_body=True")?
Original issue reported on code.google.com by hegel...@gmail.com
on 24 Aug 2012 at 1:53
GoogleCodeExporter commented
+1.
I'm using httplib2 and try to PUT large files(>5GB file). but it will stops
during processing. Is there a way to PUTing iteratively or to prevent timeout?
Original comment by hashim...@gmail.com
on 12 Mar 2013 at 9:06