Support for longer perceptive hashes
pensnarik opened this issue · 4 comments
You extension is fine but it seems it only supports 64 bits perceptive hashes. But one might use a hash with a different length. For instance, in my application I use 256 bit hashes. Therefore I cannot use this extension to build indexes on them.
64bit probably because we can natively process it on modern CPU
It's currently 64 bit because I can (ab)use inline storage for them. Postgresql itself uses 64-bit pointers, and you can just store your data right in what would be the pointer otherwise (all the <= 64-bit datatypes are implemented this way, actually).
There shouldn't be too much work to support out-of-line storage, though I'm super rusty about postgres dev stuff at the moment (see the failing tests too).
Is it possible to just use two columns to support 128bits? There would probably be a performance hit but it seems like that should work.
As I understand it, using 2 columns wouldn't provide any guarantee that the two column's representations would be stored adjacent to each other on-disk. As such, you'd not have any advantage over out-of-line storage (which is the "correct" way to do this), and it would likely take a similar amount of effort.