sudden, massive memory usage
Closed this issue · 1 comments
I haven't found a way to consistently duplicate this yet, but occasionally gln_index will suddenly, rapidly increase its memory usage - I think an edge case causes one of the dynamically resizable buffers to repeatedly double in size. It's tricky to debug because it may take 45 minutes to happen, and there's about a 1-second window before gln_index is using > 3 GB ram and my systems starts swapping aggressively.
Observed on OpenBSD/amd64.
I switched to 2-byte file hashes but still left hash table bucket sizes > 2^16, so as soon as the table grew larger than 2^16, most inserts would be likely to trigger another resize - nothing actually got split up. Fixed; going back to 4-byte hashes is still a possibility, though.