Run scrapers for period 2014 - 2015

Question

Run scrapers for period 2014 - 2015

Closed this issue 9 years ago · 2 comments

The old data ranges from 2000-01-01 until 2013-12-17. Before deploying the scrapers, we should perform a run that scrapers all articles from 2013-12-18 until the deploy day.

Answer 1 · 2015-09-18T13:22:50.000Z

This works perfect. Will run it again on final launch.

Answer 2 · 2015-09-27T08:05:17.000Z

I noticed that for the detijd scraper, a lot of articles are missing. Only the most recent articles (april 2015 - now) are fetched. Maybe this is due to some limit on the search output. I'm running it in steps for the period december 2013 - april 2015 and I document it here in case we have to redeploy some time.