Async web crawler using queue system To store web data from scrapy module via celery and rabbitmq as async Stack Django Celery Rabbitmq Scrapy