logstash-plugins/logstash-filter-json

If source is empty, then warning that target is required

robin13 opened this issue · 0 comments

If the source field is empty, then the following (misleading) warning is posted:

Parsed JSON object/hash requires a target configuration option {:source=>"[json_data]", :raw=>""}

It would be better if the warning simply said "no data in source" or similar.

  • Version: 7.11.1
  • Operating System: Ubuntu 20.04.1
  • Config File (if you have sensitive info, please remove it):
input {
    generator {
        lines => [ 
            'Mary had a little lamb'
        ]
        count => 1
    }
}

filter {
    mutate {
        add_field => { "[json_data]" => '' }
    }

    json {
        source => "[json_data]"
    }
}

output {
    stdout {
        codec => rubydebug
    }
}
  • Sample Data:
    Using generator above

  • Steps to Reproduce:

logstash-7.11.1/bin/logstash -f ./logstash.conf 

results in:

[2021-02-23T10:25:28,403][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.11.1", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc Java HotSpot(TM) 64-Bit Server VM 25.201-b09 on 1.8.0_201-b09 +indy +jit [linux-x86_64]"}
[2021-02-23T10:25:28,530][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/home/rclarke/elastic/stack/logstash-7.11.1/data/queue"}
[2021-02-23T10:25:28,549][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/home/rclarke/elastic/stack/logstash-7.11.1/data/dead_letter_queue"}
[2021-02-23T10:25:28,844][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2021-02-23T10:25:28,883][INFO ][logstash.agent           ] No persistent UUID file found. Generating new UUID {:uuid=>"016f3082-81d3-4d0e-bb24-a5b79c958a60", :path=>"/home/rclarke/elastic/stack/logstash-7.11.1/data/uuid"}
[2021-02-23T10:25:30,841][INFO ][org.reflections.Reflections] Reflections took 47 ms to scan 1 urls, producing 23 keys and 47 values 
[2021-02-23T10:25:32,063][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1000, "pipeline.sources"=>["/home/rclarke/elastic/cases/00683551/logstash.conf"], :thread=>"#<Thread:0x55c8b42f run>"}
[2021-02-23T10:25:32,913][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>0.84}
[2021-02-23T10:25:32,953][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
[2021-02-23T10:25:33,041][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2021-02-23T10:25:33,247][WARN ][logstash.filters.json    ][main][90d41e283dbbbbf33be225cfd0852dcd43791f2c1ba397160b7277f9aeb2bacd] Parsed JSON object/hash requires a target configuration option {:source=>"[json_data]", :raw=>""}
[2021-02-23T10:25:33,493][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
{
      "@version" => "1",
       "message" => "Mary had a little lamb",
    "@timestamp" => 2021-02-23T09:25:33.020Z,
      "sequence" => 0,
          "tags" => [
        [0] "_jsonparsefailure"
    ],
     "json_data" => "",
          "host" => "rcl-laptop"
}
[2021-02-23T10:25:33,664][INFO ][logstash.javapipeline    ][main] Pipeline terminated {"pipeline.id"=>"main"}