d1vanov/libquentier

New synchronization engine

Opened this issue · 2 comments

Current implementation of synchronization suffers from multiple issues, notably #187 and #188. Need a new implementation which is

  1. Durable - it shouldn't download the same things over and over with each sync attempt even if it is interrupted either by rate limit exceeding error or manually by user. It's close to impossible to make progress in the sync process otherwise.
  2. Fault tolerant. If some data item failed to be downloaded or processed and its presence is not critical for other items (i.e. any note basically), the sync shouldn't stop. It should go on but just make a note that some particular item wasn't downloaded properly so maybe it could try to re-download it during the next sync attempt.
  3. Maintainable. Current implementation is quite monstrous because when it was originally implemented years I was aware of only one kind of asynchronous processing - signal/slot based one. Needless to say, it's not the most convenient one. Future/promise style concurrency is much more convenient, especially when you add lambdas to the mix.

This is being worked on right now in branch experiment/new-synchronizer. I'm close enough to the completion of the first implementation of new synchronization engine with unit tests but integrational tests will be needed as well and it might take some additional time if I run into some issue and need to investigate and fix it.

New synchronization engine uses new local storage and depends on updated QEverCloud from branch feature/copy-on-write.