Run scrapers for period 2014 - 2015
Closed this issue · 2 comments
bartaelterman commented
The old data ranges from 2000-01-01 until 2013-12-17. Before deploying the scrapers, we should perform a run that scrapers all articles from 2013-12-18 until the deploy day.
bartaelterman commented
This works perfect. Will run it again on final launch.
bartaelterman commented
I noticed that for the detijd
scraper, a lot of articles are missing. Only the most recent articles (april 2015 - now) are fetched. Maybe this is due to some limit on the search output. I'm running it in steps for the period december 2013 - april 2015 and I document it here in case we have to redeploy some time.