RPC gracefulReload fails for mongo_replset plugin
jeevamanivel opened this issue · 0 comments
jeevamanivel commented
We are using mongo_replset plugin to forward logs to Mongo DB. When we reload fluentd config using fluentd HTTP RPC api, instead of restarting fluentd, it fails with below error
Reload URL : http://127.0.0.1:24444/api/config.gracefulReload
Config:
<system>
rpc_endpoint 127.0.0.1:24444
</system>
<source>
@type forward
@id mongo_forward
port 24224
bind 0.0.0.0
@label @LOGS
</source>
<label @LOGS>
<match mongo_forward>
@type mongo_replset
nodes HOST_PORT
database DB_NAME
collection COLLECTION_NAME
replica_set REPLICA_NAME
include_time_key false
date_keys datekey
<buffer tag>
@type file
path /fluentd/work/buf
chunk_limit_size 256m
total_limit_size 500g
compress gzip
queued_chunks_limit_size 10
flush_thread_count 10
flush_mode interval
flush_interval 2s
flush_thread_burst_interval 1.0
flush_at_shutdown true
overflow_action throw_exception
retry_max_times 3
retry_exponential_backoff_base 2
retry_type exponential_backoff
retry_timeout 24h
retry_wait 2s
retry_forever false
</buffer>
</match>
</label>
Error
2023-03-23 09:48:34 +0530 [debug]: fluent/log.rb:309:debug: preparing shutdown output plugin type=:mongo_replset plugin_id="object:a50"
2023-03-23 09:48:34 +0530 [info]: fluent/log.rb:330:info: shutting down output plugin type=:mongo_replset plugin_id="object:a50"
2023-03-23 09:48:34 +0530 [warn]: fluent/log.rb:351:warn: unexpected error while shutting down on output plugin plugin=Fluent::Plugin::MongoOutputReplset plugin_id="object:a50" error_class=NoMethodError error="undefined method `close' for nil:NilClass\n\n @client.close\n ^^^^^^"
#<Thread:0x0000029a74c9d590 C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3-x64-mingw-ucrt/lib/fluent/root_agent.rb:264 run> terminated with exception (report_on_exception is true):
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3-x64-mingw-ucrt/lib/fluent/log.rb:566:in `caller_line': undefined method `[]' for nil:NilClass (NoMethodError)
line = caller(depth+1)[0]
^^^
from C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3-x64-mingw-ucrt/lib/fluent/log.rb:467:in `dump_stacktrace'
from C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3-x64-mingw-ucrt/lib/fluent/log.rb:357:in `warn_backtrace'
from C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3-x64-mingw-ucrt/lib/fluent/root_agent.rb:287:in `rescue in block (3 levels) in shutdown'
from C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3-x64-mingw-ucrt/lib/fluent/root_agent.rb:266:in `block (3 levels) in shutdown'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/fluent-plugin-mongo-1.6.0/lib/fluent/plugin/out_mongo.rb:189:in `shutdown': undefined method `close' for nil:NilClass (NoMethodError)
@client.close
^^^^^^
from C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3-x64-mingw-ucrt/lib/fluent/root_agent.rb:280:in `block (3 levels) in shutdown'
2023-03-23 09:48:34 +0530 [debug]: fluent/log.rb:309:debug: Got Win32 event "fluentd_21228_STOP_EVENT_THREAD"
2023-03-23 09:48:34 +0530 [error]: fluent/log.rb:372:error: Failed to reload config file: undefined method `[]' for nil:NilClass
line = caller(depth+1)[0]
^^^
Unexpected error undefined method `[]' for nil:NilClass
line = caller(depth+1)[0]
^^^
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3-x64-mingw-ucrt/lib/fluent/log.rb:566:in `caller_line'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3-x64-mingw-ucrt/lib/fluent/log.rb:467:in `dump_stacktrace'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3-x64-mingw-ucrt/lib/fluent/log.rb:357:in `warn_backtrace'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3-x64-mingw-ucrt/lib/fluent/root_agent.rb:287:in `rescue in block (3 levels) in shutdown'
C:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/fluentd-1.15.3-x64-mingw-ucrt/lib/fluent/root_agent.rb:266:in `block (3 levels) in shutdown'