method received not called in logstash 5.x in favor of the inherited multi_receive_encoded
jsvd opened this issue · 5 comments
Since this plugin inherits from logstash-output-file,
the multi_receive_encoded from the file output gets called instead of this own plugin's receive
This means this plugin behaves like logstash-output-file in logstash 5.x, making it useless. For more reports see:
https://discuss.elastic.co/t/elasticsearch-logstash-input-to-csv-output-mapping-question/65837
https://discuss.elastic.co/t/csv-output-plugin-prints-wrong-stuff/65574
https://discuss.elastic.co/t/debugging-csv-output/64901
https://discuss.elastic.co/t/export-from-elasticsearch-to-csv-problem-where-i-am-wrong/65558
HI!
We could modify csv.rb file and replace receive(event) fonction with multi_receive_encoded(encoded) fonction ?
def multi_receive_encoded(encoded)
encoded.each do |event,data|
path = event.sprintf(@path)
fd = open(path)
csv_values = @fields.map {|name| get_value(name, event)}
fd.write(csv_values.to_csv(@csv_options))
flush(fd)
close_stale_files
end
end
@nachonam would you mind submitting a PR with that change? we already have tests in place, they should validate if the change is good.
Yes! I did some tests in my local env... and all it's ok for me
PR done
I have a better implementation of a fix for this than the closed PR above. Its based on the File output's method and therefore suits multi-threading better.
I will add a PR shortly.