d1vanov/libquentier

Change the way resources binary data is stored in the local storage: move them out of the database to separate files

Closed this issue · 2 comments

As it became clear in d1vanov/quentier#206, select queries become very slow if resources table is involved even if heavy binary data from this table is not actually selected. It seems to be caused by SQLite internal architecture. The way to avoid this performance issue is to not store the heavy binary data within the same table as other not so heavy data which is selected.

So need to split the existing resources table into two parts: resource metadata and resource binary data. Probably it'd be enough for the latter table to only contain dataBody and alternateDataBody columns from the current Resources table, recognitionDataBody shouldn't be heavy enough + I recall it is used in note search queries so it shouldn't belong to the slow table.

For now this issue is blocked by d1vanov/quentier#213.

In fact, I might want to go one step further and store resource binary data as plain files instead of within the database - this page suggests large enough blobs are better stored outside the database for acceptable read performance.

Finally finished, merged to development branch.