JasonSanford/gitspatial

Catch Memory Errors

Closed this issue · 0 comments

OMG, 60MB GeoJSON

INFO 2013-09-26 14:06:57,443 gitspatial.tasks Setting feature set sync status as syncing: mikelmaron/kenya-election-data/data/wards.geojson
INFO 2013-09-26 14:06:57,459 requests.packages.urllib3.connectionpool Starting new HTTPS connection (1): api.github.com
INFO 2013-09-26 14:06:57,519 requests.packages.urllib3.connectionpool Starting new HTTPS connection (1): api.github.com
INFO 2013-09-26 14:06:57,582 requests.packages.urllib3.connectionpool Starting new HTTPS connection (1): api.github.com
INFO 2013-09-26 14:06:58,494 requests.packages.urllib3.connectionpool Starting new HTTPS connection (1): api.github.com
Traceback (most recent call last):
File "", line 1, in
File "/home/gitspatial/apps/gitspatial/venv/local/lib/python2.7/site-packages/celery/app/task.py", line 330, in call
return self.run(_args, *_kwargs)
File "/home/gitspatial/apps/gitspatial/gitspatial/tasks.py", line 128, in get_feature_set_features
content = _get_blob(github, feature_set)
File "/home/gitspatial/apps/gitspatial/gitspatial/tasks.py", line 177, in _get_blob
blob_request = github.get(blob_url)
File "/home/gitspatial/apps/gitspatial/gitspatial/github.py", line 17, in get
return requests.get(self._base_uri + method, headers=self._headers)
File "/home/gitspatial/apps/gitspatial/venv/local/lib/python2.7/site-packages/requests/api.py", line 55, in get
return request('get', url, *_kwargs)
File "/home/gitspatial/apps/gitspatial/venv/local/lib/python2.7/site-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, *_kwargs)
File "/home/gitspatial/apps/gitspatial/venv/local/lib/python2.7/site-packages/requests/sessions.py", line 335, in request
resp = self.send(prep, *_send_kwargs)
File "/home/gitspatial/apps/gitspatial/venv/local/lib/python2.7/site-packages/requests/sessions.py", line 438, in send
r = adapter.send(request, *_kwargs)
File "/home/gitspatial/apps/gitspatial/venv/local/lib/python2.7/site-packages/requests/adapters.py", line 340, in send
r.content
File "/home/gitspatial/apps/gitspatial/venv/local/lib/python2.7/site-packages/requests/models.py", line 594, in content
self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes()
MemoryError