CycodeLabs/raven

Enable parallel run of indexer and downloader

Closed this issue · 0 comments

They should communicate using Redis.
I had two ideas -

  1. Better - Use Redis set as a queue. There will be a set (or two - one for actions and one for workflows) that contains items to be indexed. Once they scan them, they will remove the item
  2. The indexer will get last items in download_history sets every time their length is getting bigger.

Blocked by -
This issue #1