pinojs/pino-elasticsearch

Connection closing

Closed this issue · 8 comments

Am I right the the ElasticSearch connections are never closed? I see lots of active clients at ElasticSearch (value of nodes.http.current_open) and no trace that connections are closed using pino-elasticsearch. Since the client is not accessible outside it doesn't seem possible to close a connection by myself so how should they get closed?

The client relies on the internal http agent to handle connections. If a connection it's not used, after a while it will be closed automatically.

I have connections active for hours now. Wouldn't be better to close connections sooner?
I don't know how important that is when using ElasticSearch though.

You can configure for how much time an idle socket should be kept alive. But if you are sending data continuously it won't be closed.
It a good thing that connections are not closed often tho, opening a new one is costly.
Reusing old connections is more performant.

If you have a huge number of connections that you aren't expecting (by default the client opens up to 512 sockets), it might be worth investigating how many sources are connected to Elasticsearch (which can handle thousands of connections at the same time).

I checked and the connections (88) are active for 3 days now . I haven't changed any Elasticsearch setting related to this so this is by default. These were created by me when implementing pino-elasticsearch. I'll reset these manually

How could I configure idle sockets keep alive? Maybe you mean the network.tcp.keep_idle ElasticSearch setting?

The sockets are being kept alive by default by the client.

By client you mean the @elastic/elasticsearch client, right? Since Friday I've even restarted the OS running the app with pino-elasticsearch.
Sorry but it's not clear to me how are these connections still alive and how to close them after some time (like 1 day)

I'm going in a different direction. Thank you!