TeamHG-Memex/scrapy-rotating-proxies

In distributed crawling environment

vijay-axis opened this issue · 0 comments

vijay-axis commented

Can this be used in distributed crawling enviroment. Or is there any other solution for crawling in distributed environment