Download image script aborts as soon as an image cannot be downloaded
yayayayaka opened this issue · 0 comments
yayayayaka commented
When an image cannot be downloaded (for example when the hosting server has gone extinct) ,the script aborts with an AssertionError
without downloading the rest of the images.
Stack Trace:
$ python download_images.py --no-thumbnails
Skipping 872 already-downloaded images
Downloading 73 new images...
Traceback (most recent call last):
File "download_images.py", line 63, in <module>
download_images()
File "/home/naka/.local/share/virtualenvs/matrix-archive-j2607j-8/lib/python3.6/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/home/naka/.local/share/virtualenvs/matrix-archive-j2607j-8/lib/python3.6/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/naka/.local/share/virtualenvs/matrix-archive-j2607j-8/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/naka/.local/share/virtualenvs/matrix-archive-j2607j-8/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "download_images.py", line 59, in download_images
run_downloads(new_messages, download_dir, prefer_thumbnails=thumbnails)
File "download_images.py", line 23, in run_downloads
assert res.status_code == 200
AssertionError
Rather than asserting a 200
response, I'd propose catching the requests.exceptions.RequestException
and continue downloading the next images.