Feature Request: "--retries" option
EpicLPer opened this issue · 0 comments
EpicLPer commented
Heya,
Stumbled upon this script while trying to scrape a .onion site (with a bit of a workaround).
So far it's working okay, but due to the nature of Tor it sometimes happens that some sites report a 404 or other error codes.
For such cases a "--retries" option would be handy so the script tries by itself to index that specific URL again for x
given attempts :)
Thanks!