API for synchronization process introspection

[d1vanov]

Some issues that can arise during the synchronization are hard to debug using only debug or even trace logs. Full synchronization can produce gigabytes worth of debug logs on large enough accounts and extracting useful information from these logs can be hard.

It would be nice to have some API which could provide some kind of introspection into the running (or last performed) sync process. One example of a problem which is hard to debug without such API: during a full sync a 1000+ notes were downloaded in a reasonable time but then the sync continues to run seemingly indefinitely trying to download 2-3 remaining notes. From logs one can't figure out the guids of these notes, at least not easily. Even worse, one cannot easily tell whether something got stuck into some kind of deadlock and waiting more won't be fruitful.

From developer's perspective an issue like that could be investigated by extracting and examining some stats from the sync process' state as well as available detailed information about problematic parts. Then, armed with this information, one can go and grep some info from large logs.

I imagine creating something like this won't be very easy so it's better to start small: create an API which would expose existing stats collected during the sync. Perhaps organize the existing stats somewhat more logically. Add new stats which are currently not collected anywhere. The API should return the stats in some generic enough form like a json of arbitrary structure. The results of these API calls wouldn't be stable and would represent no more than a reflection of some library internals.

The next step could be defining a way to "ask questions" to the sync state i.e. add a way to pass commands with arguments to introspection requests, like get_note_info guid=<guid>. The response should contain info on the specified entity - its state (already synced, downloading, pending download etc.), probably some sync stats (download time) and anything else useful.