LevelDB native memory consumption is increasing

[kaymakyavuz]

We are using leveldbjni-linux64 dependency in our backends which implement only read operations (we are filling the DB beforehand in a separate process). The DB holds about 200mil keys and recieves +300 requests per second. We are seeing an unbounded increase in native memory consumption. Pmap shows +2GB of SST files and +11GB anonymous, when working-set is about 6 GB. I know that under pressure should leveldb give these files back. But that does not happen in our case.

Setting: Cache size: 100MB, max_open_files=default

[Krishna7557]

@kaymakyavuz, we also face similar issue while running LevelDB on Linux server. When we look at pmap we could see loy anon objects consuming memory. Did you find what is the cause for this?

Regards,
Krishna

[kaymakyavuz]

Hi @Krishna7557 , the only solution I found was to tweak the unix env, which I did not test. I have stopped looking for a solution, just massively (x3) scaled deployment. For the time being it is relatively better but not ideal. I am considering alternative options