Fluentd Docker Container Overwhelmed with Plugin Error Messages
amrubal01 opened this issue · 1 comments
amrubal01 commented
Hi all!
Describe the bug
In my logs, both output in stdout (make logs) and into files, there is an overwhelming amount of fluentd container logs regarding plug-in issues. I am running the fluentd(v1.16.2) docker image from DockerHub. Error logs attached at bottom of issue.
I noticed that a comment on issue #3096 had similar error logs but I am not usig nginx so I am unsure if it is completely related.
Expected Behavior
No error logs of this kind.
Your Environment
Fluentd v1.16.2 (Docker image pulled from DockerHub)
Docker Desktop 4.25.0
Your Configuration
<source>
@type forward
@id input1
@label @mainstream
port 24224
</source>
<filter **>
@type stdout
</filter>
<label @mainstream>
<match docker.**>
@type file
@id output_docker1
path /fluentd/log/docker.*.log
symlink_path /fluentd/log/docker.log
append true
time_slice_format %Y%m%d
timekey_wait 1m
time_format %Y%m%dT%H%M%S%z
</match>
<match **>
@type file
@id output1
path /fluentd/log/data.*.log
symlink_path /fluentd/log/data.log
append true
time_slice_format %Y%m%d
timekey_wait 10m
time_format %Y%m%dT%H%M%S%z
</match>
</label>
Your Error Log
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/output.rb:915:in `metadata'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/output.rb:1042:in `block in handle_stream_with_custom_format'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/event.rb:110:in `each'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/output.rb:1041:in `handle_stream_with_custom_format'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/output.rb:969:in `execute_chunking'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/output.rb:897:in `emit_buffered'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/event_router.rb:115:in `emit_stream'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/event_router.rb:106:in `emit'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:355:in `on_message'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:226:in `block in handle_connection'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:263:in `block (3 levels) in read_messages'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:262:in `feed_each'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:262:in `block (2 levels) in read_messages'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:271:in `block in read_messages'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:640:in `on_read_without_connection'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/cool.io-1.8.0/lib/cool.io/io.rb:123:in `on_readable'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/cool.io-1.8.0/lib/cool.io/io.rb:186:in `on_readable'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/cool.io-1.8.0/lib/cool.io/loop.rb:88:in `run_once'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/cool.io-1.8.0/lib/cool.io/loop.rb:88:in `run'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/event_loop.rb:93:in `block in start'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [warn]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [error]: #0 [input1] unexpected error on reading data host="192.168.65.1" port=43682 error_class=ArgumentError error="record must be a Hash: String"
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [error]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/output.rb:915:in `metadata'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [error]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/output.rb:1042:in `block in handle_stream_with_custom_format'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [error]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/event.rb:110:in `each'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [error]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/output.rb:1041:in `handle_stream_with_custom_format'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [error]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/output.rb:969:in `execute_chunking'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [error]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/output.rb:897:in `emit_buffered'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [error]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/event_router.rb:115:in `emit_stream'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [error]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/event_router.rb:106:in `emit'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [error]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:355:in `on_message'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [error]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:226:in `block in handle_connection'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [error]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:263:in `block (3 levels) in read_messages'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [error]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:262:in `feed_each'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [error]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:262:in `block (2 levels) in read_messages'
cluster-fluentd-1 | 2023-10-25 16:32:18 +0000 [error]: #0 /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:271:in `block in read_messages'