muddydixon/fluent-plugin-datacalculator

string class -> to_i/to_f sapport

Closed this issue · 2 comments

ftpd xfer.log in in_tail plugin
value is String Class

fluentd log

2013-07-10 14:56:24 +0900 [warn]: emit transaction failed error="String can't be coerced into Fixnum"
2013-07-10 14:56:24 +0900 [warn]: /etc/td-agent/plugin/out_datacalculator.rb:316:in `+'

source change
c[index] += execFunc(nil, record, inkeys, formula).to_i

Are there any other Countermeasure?

適当な英語ですみません
ftpデーモンのxfer.logをin_tailで読み込んで、一定間隔毎の転送サイズ合計を
求めようとしたのですが、文字列Classのため、本プラグインがエラーとなりました
暫定としてto_iメソッドにより数値Classに変更しましたが
別な対策ありますか?

遅くなってしまいすみません。
もう少し情報をいただけますでしょうか。

具体的には、どのような設定で、どのようなmessageを流したときに上記のエラーが出てきたかを教えていただけると助かります。
実際のデータでなくともよいのですが、つかめるようなデータを示していただけますでしょうか。

td-agent 1.1.14-0
td-agent config file

<source>
  type tail
  format /^(?<time>\w+ \w+ \d\d \d+:\d+:\d+ \d+) (?<trans_time>\d+) (?<ipaddress>[^ ]*) (?<size>\d+) (?<filename>[^ ]+) (?<type>\w) (?<special_action>\w) (?<direction>\w) (?<access_mode>\w) (?<user>[^ ]*) (?<service>[^ ]*) (?<auth_method>\d) (?<auth_user_id>[^ ]+) (?<status>\w)$/
  path /var/tmp/xfer.log
  tag ftp.xfer_log
  time_format %a %b %d %H:%M:%S %Y
  pos_file /var/tmp/xfer.log.pos
</source>
<match ftp.xfer_log>
    type      datacalculator
    tag       test.ftp_size
    unit      minute
    aggregate tag
    formulas  total = size
</match>
<match test.ftp_size>
  type file
  path /var/tmp/ftp.log
</match>

input massage(proftpd xferlog)

Wed Jul 10 04:02:32 2013 0 127.0.0.1 36 /home/ftp/yyyy/incoming/hogehoge b _ d r yyyy ftp 0 * c
Wed Jul 10 04:03:01 2013 0 000.000.000.000 8052 /home/ftp/xxxx/incoming/fugafuga b _ o r xxxx ftp 0 * c

td-agent log file

2013-07-17 09:40:12 +0900 [info]: following tail of /var/tmp/xfer.log
2013-07-17 09:40:12 +0900 [warn]: emit transaction failed  error="String can't be coerced into Fixnum"
  2013-07-17 09:40:12 +0900 [warn]: /etc/td-agent/plugin/out_datacalculator.rb:316:in `+'
  2013-07-17 09:40:12 +0900 [warn]: /etc/td-agent/plugin/out_datacalculator.rb:316:in `block (2 levels) in emit_single_tag'
  2013-07-17 09:40:12 +0900 [warn]: /etc/td-agent/plugin/out_datacalculator.rb:313:in `each'
  2013-07-17 09:40:12 +0900 [warn]: /etc/td-agent/plugin/out_datacalculator.rb:313:in `block in emit_single_tag'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/event.rb:107:in `call'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/event.rb:107:in `block in each'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/event.rb:106:in `each'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/event.rb:106:in `each'
  2013-07-17 09:40:12 +0900 [warn]: /etc/td-agent/plugin/out_datacalculator.rb:310:in `emit_single_tag'
  2013-07-17 09:40:12 +0900 [warn]: /etc/td-agent/plugin/out_datacalculator.rb:276:in `emit'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/output.rb:35:in `next'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/buffer.rb:170:in `block in emit'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/1.9.1/monitor.rb:211:in `mon_synchronize'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/buffer.rb:166:in `emit'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/output.rb:509:in `block in emit'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/event.rb:107:in `call'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/event.rb:107:in `block in each'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/event.rb:106:in `each'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/event.rb:106:in `each'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/output.rb:499:in `emit'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/output.rb:35:in `next'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/plugin/out_copy.rb:67:in `emit'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/match.rb:38:in `emit'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/engine.rb:134:in `emit_stream'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/plugin/in_tail.rb:106:in `receive_lines'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/plugin/in_tail.rb:324:in `call'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/plugin/in_tail.rb:324:in `on_notify'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/plugin/in_tail.rb:155:in `on_notify'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/plugin/in_tail.rb:260:in `call'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/plugin/in_tail.rb:260:in `on_change'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/cool.io-1.1.0/lib/cool.io/loop.rb:96:in `run_once'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/cool.io-1.1.0/lib/cool.io/loop.rb:96:in `run'
  2013-07-17 09:40:12 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.35/lib/fluent/plugin/in_tail.rb:83:in `run'