scraper-site-manga

This repository scrap information from manga teams websites to feed https://fuckjapscan.com/ website.

Project setup

conda env create -f environment.yml

Add this lines to scrapy settings

CONNECTION_STRING = 'sqlite:////path/to/database/fuckjapscan.db'

IMAGES_STORE = '/path/to/images/'

scrapy crawl spider_name

For example if you want to run FuryoSquad spider :
scrapy crawl furyosquad

scrapyd-deploy to update egg.
scrapyd-client schedule -p sitemanga \* to run spiders.