Optimize BLOB storage

[tciuro]

Some users need/want to store BLOBs in the database. The problem with that is a potential lack of performance since it could cause paging... and causing a major I/O bottleneck. A nice article can be found on SQLite's website:

http://www.sqlite.org/intern-v-extern-blob.html

I think that a great way to optimize NanoStore would be to treat BLOBs in a special manner. This is what I'm thinking:

• Do not store the BLOB component(s) found in the NSFKeyedArchive BLOB. Instead, keep a separate list of these BLOBs
• During save, the BLOB-less NSFKeyedArchive gets stored in the NSFKeys table and the related BLOBs are stored in a 3rd table called NSFBLOBs. This table contains three columns: ROWID, Key, Key path and BLOB.
• During object retrieval using NSFReturnObjects, the NSFKeyedArchive BLOB would be coalesced with its NSFBLOBs-related rows. If NSFSearch contains a set of attributes specified via attributesToBeReturned, then the partial object would be built accordingly.

The save process would perhaps be slightly slower due to separation of the BLOBs from the NSFKeyedArchive BLOB. However, reads would generally be faster and the memory footprint kept lower because BLOBs wouldn't be juggled around while dealing with the NSFValues table.