msimerson/safe-log-reader

compare to filebeat

Closed this issue · 2 comments

jaredj commented

possibly silly question. I recall we created this project because logstash was so heavy. Now there is filebeat. How does this project compare to Filebeat?

msimerson commented

Filebeat is much better than logstash, I’m using it in a few places now. Much of the smarts can be done in ES now with the fairly simple Go app running on the clients.

You are forgetting the other major reason that we created this library: to permit the postfix processor to aggregate the output of many related log messages into a single postfix document. That’s still dramatically more efficient for ES and humans than spewing the logs into ES and then aggregating them later by postfix IDs.

jaredj commented

@msimerson ah yes! I may be experimenting with another application like that, this tool might be helpful again. Although we have a centralized logstash server that we have filebeat feeding into... it may be possible to use the aggregate filter on the centralized logstash server instead. I haven't looked closely at it yet

Thanks for refreshing my memory :)