open-telemetry/opentelemetry-collector

[exporter/otlphttp] Log levels: WARN and ERROR do not work for the OTLP exporter to collect internal telemetry logs to monitor the collector

Opened this issue · 2 comments

Describe the bug

"There is a problem when using Internal Telemetry to send OTEL collector internal logs to the OTLP endpoint using the OTLP Exporter. Logs are being sent to the endpoint, but the log level does not work as expected. If I use "WARN" or "ERROR," I still receive logs of INFO, DEBUG,WARN and ERROR. This is overwhelming my end system with a large volume of logs. I want to see the logs at the specified log level."

  logs: 
    level: warn
    encoding: json
    processors:
      - batch:
          exporter:
            otlp:
              protocol: http/protobuf
              endpoint: 
              headers:
                Authorization:

Steps to reproduce

Please use the configuration below. I am trying to send OTEL collector logs to my ActiveGate in Dynatrace to monitor the collector. The logs are being sent without any problem, but you will find that the LogLevel filter does not work. It only works for DEBUG.

service:

 pipelines:
  logs:
    receivers: [otlp]
    processors: [memory_limiter,batch, resourcedetection]
    exporters: [otlp]
telemetry:
    resource:
      service.name: XXXX
      deployment.environment: XXXX

  metrics:
    level: normal
    readers:
      - periodic:
          interval: 10000
          exporter:
            otlp:
              protocol: http/protobuf
              endpoint: XXXX
              headers:
                Authorization: XXXX
                   
  logs: 
    level: warn
    encoding: json
    processors:
      - batch:
          exporter:
            otlp:
              protocol: http/protobuf
              endpoint: XXXX
              headers:
                Authorization: XXXX

What did you expect to see?
Logs Level should filter the logs

What did you see instead?
I want to see the logs at the specified log level, LogLevel : Warn or Error filter doesnt work.

What version did you use?
0.113.0

What config did you use?

Environment
Microsoft Windows [Version 10.0.19045.5011]

Additional context
https://github.com/open-telemetry/opentelemetry.io/blob/main/content/en/docs/collector/internal-telemetry.md#configure-internal-logs

https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/otlphttpexporter/README.md

With Level : WARN

OTLP Dynatrace : consumes my storage.

image

I was facing the same issue using 0.113.0 and 0.114.0 but it is working as expected now using version 0.115.0 on Linux