Has anyone hit a limit with this API?
Closed this issue · 2 comments
igalci commented
The scraper seems a bit aggressive with Async. Has anyone encountered problems downloading hundreds of pages? How often can I run this script?
d3an commented
The scraper currently defaults to a spaced-sequential method due to Finviz's API limits. You could still use async, but users were reporting missing pages in the past. Consequently, the spaced-sequential request method is your best option to achieve data integrity.
igalci commented
oh yes, you are right. It uses sequential scrape.
So downloading ~150 pages, how often can I do that? once per hour is fine? can I do it more frequently?