tombh/dharma-api

Spiders coming up short locally

danbartlett opened this issue · 1 comments

When I run all the spiders locally I get about 8000 talks. The live site lists around 22,000 - what am I missing?

Some of the scrapers only fetch the most recent talks. I think it's just to save time. I wonder how big a dump file would be?