Examples for web scraping and crawling Useful resources Basic Knowledge CSS Selectors HTML DOM Core lxml cssselect (required for lxml if css selectors are used) requests Extra splash ratelimiter requests-cache tablib Some day These are used for more complicated crawlers. celery rabbitmq scrapy