logstash excesive memory usage
litinoveweedle opened this issue · 2 comments
I did installed on the ELK 7.10 (basic license) both synesis lite v1.1.0 and elastiflow 4.0.1. I did set logstash HEAP to 4GB, elastisearch to 32GB on physical server with 64GB of RAM and swap disabled. Nothing else is installed either on ELK or on the server itself.
VIRT usage for elasticsearch raised to 42GB, logstash to 22GB effectively eating whole memory, with other processes OOMs, even when ELK is currently doing nothing, as I do not route any logs / netflow to the server yet! I had to limit elasticsearch to 24GB and now it look like this (again just quietly sitting):
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27060 logstash 20 0 22.4g 5.0g 26360 S 11.9 8.0 26:51.28 java
26502 elastic+ 20 0 34.0g 25.5g 29664 S 0.3 40.5 4:14.21 java
I really do not understand usage of 20GB of logstash RAM, do you please have any explanation or suggestion? Is there anything I could do to limit logstash from such excessive memory usage?
Sorry for accidentally closing the issue :-o
Elasticsearch should not be configured for more than 31GB heap. Also, the readme specifies Elastic Stack 7.1.x.
I can't comment on Logstash without much more information, especially logs. The primary reason that this repo isn't really maintained anymore is that I have been actively moving away from Logstash.