logstash fails to start if template cache file is empty

[abeeson]

After a reboot of our logstash host which failed (unsure why, sorry) our template cache file remained, but was blank. Logstash then failed to start, throwing errors as it was unable to run the file through the JSON parser:

[ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"RuntimeError", :message=>"LogStash::Codecs::Netflow: templates cache file could not be read @ (/ipfix//ipfix_templates.cache: JSON::ParserError A JSON text must at least contain two octets!)", :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-codec-netflow-3.11.2/lib/logstash/codecs/netflow.rb:427:in load_templates_cache'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-codec-netflow-3.11.2/lib/logstash/codecs/netflow.rb:90:in register'", "/usr/share/logstash/logstash-core/lib/logstash/codecs/base.rb:20:in initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-codec-netflow-3.11.2/lib/logstash/codecs/netflow.rb:54:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/plugins/plugin_factory.rb:89:in plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:112:in plugin'", "(eval):8:in <eval>'", "org/jruby/RubyKernel.java:994:in eval'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:84:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:169:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:315:in block in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:312:in block in converge_state'", "org/jruby/RubyArray.java:1734:in each'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:299:in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:in block in converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:in converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:348:in block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in block in initialize'"]}

It would be good to have an "empty file" check before this to help eliminate what i assume could be a relatively common issue in the case of unexpected exits etc.

• Version: Logstash 6.2.3
• Operating System: RHEL 7
• Config File (if you have sensitive info, please remove it):
  input {
  udp {
  id => "ipfix-in"
  port => 2055
  receive_buffer_bytes => 33554432
  codec => netflow {
  cache_save_path => "/ipfix/"
  }
  }
  }
• Sample Data: n/a
• Steps to Reproduce:

1. Create a blank file in the expected cache save path called ipfix_templates.cache
2. Restart logstash
3. Observe crash as above

[jorritfolmer]

Can you check if this fixes the issue?

--- a/lib/logstash/codecs/netflow.rb
+++ b/lib/logstash/codecs/netflow.rb
@@ -77,7 +77,7 @@ class LogStash::Codecs::Netflow < LogStash::Codecs::Base
     if @cache_save_path
       if @versions.include?(9)
         cache_save_file_netflow = "#{@cache_save_path}/netflow_templates.cache"
-        if File.exists?(cache_save_file_netflow)
+        if File.size?(cache_save_file_netflow)
           raise "#{self.class.name}: Template cache file #{cache_save_file_netflow} not writable" unless Fil
           @netflow_templates_cache = load_templates_cache("#{@cache_save_path}/netflow_templates.cache")

[abeeson]

@jorritfolmer that didn't fix it, but I noticed your code only changed it for v9, whereas this if for v10, so i made that same change there (line 88 on mine, potentially line 87 for you, i think my current version might have an extra line of comments above), and it looks to be functional.

I've also repeated the test with a missing cache file, and that works as well, so I'd say that's a decent fix!

[yaauie]

Thanks for reporting and providing a patch. I'll get a PR together shortly.

[abeeson]

@yaauie I'm happy to sort the PR if it helps save you some trouble, it was @jorritfolmer that did the work :)