cbanack/comic-vine-scraper

v2 scraper thoughts

Opened this issue · 0 comments

Think if you want to do a rework of the scraper.
Make it more automatic to run in the background, ruled by a slow rate limit would be the way to go.

Have the scraper should automatically search and take the best automatic hit and save it as a temp match, waiting for approval.
Metadata for the match should be downloaded as a whole already so the user can approve at a faster speed in the end.

This removes the wait time and search time of manually matching that we have now. Even with the automatic matching, it still takes time. This time could have been done in the background when ComicRack is idle or something.

Really don't like the idea that it takes so long to get the info you want on larger collections. You never really catch up in the end cause of this at times.

Technically it could also be an external script that you can link to the comicrack pluging so they can interact. Allowing for automatic temp matching without comicrack needing to run.

If it where external with a interactions linked to comicrack then you could build on it with plugins.
Build up the comicvine scraper as its own plugin, as a default. Allowing users to maybe make other plugins for other sites, sure the manga/doujinshi community would love that too.