Save and return total number of found urls

[joscul]

Right now we cap full text results at 10M top urls. But we should also store the number of total urls found while indexing.

[joscul]

Index file format

8 bytes number of keys (n)
8 * n bytes keys
8 * n bytes positions
8 * n bytes lengths (len(k) number of records for key k)
[Data Records]

Data records are structured like this:
8 bytes total number of results
len(k) * 12 bytes record data where first 8 bytes is the unsigned long value and last 4 bytes are the unsigned integer score.