/scrawl

a url rule scrawler using proxy_pool to avoid ip back list.

Primary LanguagePython

proxy_pool

a crawler to get proxy list from cn-proxy. used for spider to crawl website via different proxy.