Datenschule/jedeschule-scraper

move scraper code out of run.py

Closed this issue · 1 comments

knutator2 commented

the code to get the data of hamburg, nrw and mecklenburg vorpommern should not be part of the run.py right now. There should be a file for each federal state I think there are 2 possibilities how to structure the crawlers:

  1. save all crawlers in the spiders directory regardless if it is a real spider or just some code downloading data from the web
  2. split spiders and more simple crawlers. Spiders go into the spiders directory all other scrapers in another
knutator2 commented

duplicate with #42