Filtering logs based on its level - do not send DEBUG logs to Elasticsearch

[marcelobp]

Hi all,

I currently have an Openshift configured with Fluentd sending logs to Elastisearch and it is working fine.

I would like to filter theses logs, sending to ES only logs from WARNING level and preventing of sending DEBUG logs.

I have add @log_level "#{ENV['ES_LOG_LEVEL'] || 'warn'}" to both output-es-config.conf and output-es-ops-config.conf however the DEBUG logs still being sent to ES. Is there anything I am missing, please?

Appreciate any of your help.

Thanks,
Marcelo

[jcantrill]

I have add @log_level "#{ENV['ES_LOG_LEVEL'] || 'warn'}" to both output-es-config.conf and output-es-ops-config.conf however the DEBUG logs still being sent to ES. Is there anything I am missing, please?

I would recommend you read the fluent documentation about filters, the snippet you include here speaks to how one might configure the log level of a given fluent plugin or possibly fluent overall. The stack collects logs from an indeterminate number of services that all can log in their own format. This means "debug" for one service may not be the same for another. Additionally, the log level may be represented differently in the actual log message.

There is no one answer to your question without knowing the log format and trying to apply that to a specific service