Full-text search
vladimiry opened this issue · 5 comments
The time has come to make it happen. I've started the research and playing around already.
I tried to keep the search index close to the database but it's not going to work well since the index is 3-5x times bigger than the database itself. So the full database serialization takes time, besides the main process might get blocked by expensive serialization/indexing/search actions. So going to try maintaining all the expensive indexing operations in a separate process and save the index in an individual encrypted file like database-index.bin.
Since indexing works quite fast I'm also considering not persisting the search index at all but just spawn the separate process which will build the index on app start and keep it up to date while the app is started, so completely in memory only. The implementation would be much simpler and somehow more secure (no addition data persisting).
The initial implementation is going to be released soon.
CC @anothernewb