fluent/fluent-plugin-webhdfs

getaddrinfo: Name or service not known

loveshell opened this issue · 3 comments

loveshell commented

for my example:

  <source>
    type forward
  </source>
    <match apache.access>
      type webhdfs
      host 192.168.1.12
      port 50070
      path /weblog/access_%Y%m%d.log
      username hdfs
      flush_interval 10s
  </match>

error:

2014-07-01 17:33:59 +0800 [trace]: plugin/in_forward.rb:245:on_close: closed fluent socket object_id=15140840
2014-07-01 17:34:01 +0800 [warn]: plugin/out_webhdfs.rb:207:rescue in write: failed to communicate hdfs cluster, path: /weblog/access_20140701.log
2014-07-01 17:34:01 +0800 [warn]: fluent/output.rb:343:rescue in try_flush: temporarily failed to flush the buffer. next_retry=2014-07-01 17:34:02 +0800 error_class="SocketError" error="getaddrinfo: Name or service not known" instance=19091100
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/1.9.1/net/http.rb:763:in `initialize'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/1.9.1/net/http.rb:763:in `open'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/1.9.1/net/http.rb:763:in `block in connect'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/1.9.1/timeout.rb:69:in `timeout'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/1.9.1/timeout.rb:100:in `timeout'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/1.9.1/net/http.rb:763:in `connect'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/1.9.1/net/http.rb:756:in `do_start'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/1.9.1/net/http.rb:745:in `start'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/1.9.1/net/http.rb:1285:in `request'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/1.9.1/net/http.rb:1265:in `send_request'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/webhdfs-0.5.5/lib/webhdfs/client_v1.rb:281:in `request'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/webhdfs-0.5.5/lib/webhdfs/client_v1.rb:242:in `operate_requests'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/webhdfs-0.5.5/lib/webhdfs/client_v1.rb:45:in `create'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluent-plugin-webhdfs-0.2.2/lib/fluent/plugin/out_webhdfs.rb:189:in `rescue in send_data'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluent-plugin-webhdfs-0.2.2/lib/fluent/plugin/out_webhdfs.rb:186:in `send_data'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluent-plugin-webhdfs-0.2.2/lib/fluent/plugin/out_webhdfs.rb:205:in `write'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.50/lib/fluent/buffer.rb:296:in `write_chunk'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.50/lib/fluent/buffer.rb:276:in `pop'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.50/lib/fluent/output.rb:310:in `try_flush'
  2014-07-01 17:34:01 +0800 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.50/lib/fluent/output.rb:132:in `run'
loveshell commented

hadoop log：

PriviledgedActionException as:hdfs (auth:SIMPLE) cause:java.io.FileNotFoundException: File /weblog/access_20140702.log not found.
loveshell commented

and i touch the file ,file still empty

repeatedly commented

'getaddrinfo: Name or service not known' is network environment issue, not plugin issue.
Fluentd has retry feature so it seems no problem.