mattgodbolt/zindex

Sparse line indexing

mattgodbolt opened this issue · 2 comments

mattgodbolt commented

Maybe we should store line lumbers in the AccessPoints so we can random access by line number without a full LineIndex (which can be big in some cases).

sbendiola commented

Are you thinking about using the line #s to support binary search in zq?

I had some issues with large indexes that were many GBs for a simple increasing sequence. I now just manually query the min max from lineoffsets and binary search. I did notice #44 and will provide a patch

mattgodbolt commented

that was the thought yeah. Mainly..whatever though :) I resurrected zindex for a local set of presentations on "how gzip works" but I don't have a (current) use case for it. I may need some random access to binary files, but...really you're more likely to know what you want! Pretty much anything you want just send me a PR :)