Datafable/epu-index

Store log of successful scraper job

Closed this issue · 1 comments

bartaelterman commented

The web application needs to know how many scrapers ran successfully in order to be able to calculate the epu index for a day. How shall we store that information?

@peterdesmet @niconoe any ideas?

bartaelterman commented
  • Model JournalScraped added (c8c5097).
  • Added close_spider method to the pipeline, writing one line for every day and journal scraped.