Invalid data stream configuration when using dlq_custom_codes option
ckauf opened this issue · 0 comments
Logstash information:
Please include the following information:
-
Logstash version (e.g.
bin/logstash --version
)
8.7.1 -
Logstash installation source (e.g. built from source, with a package manager: DEB/RPM, expanded from tar or zip archive, docker)
tar.gz -
How is Logstash being run (e.g. as a service/service manager: systemd, upstart, etc. Via command line, docker/kubernetes)
via CLI -
How was the Logstash Plugin installed
Bundled
JVM (e.g. java -version
):
Bundled
OS version (uname -a
if on a Unix-like system):
MacOS 13.4
Description of the problem including expected versus actual behavior:
Actual behavior: When adding dlq_custom_codes
to an output with data_stream => true
the pipeline does not start but instead displays the following error message:
Invalid data stream configuration, the following parameters are not supported: {"dlq_custom_codes"=>[401]}
Expected behavior: Configuring custom codes works with Data Streams
Steps to reproduce:
Create a pipeline config that looks like this:
input {
generator {}
}
output {
elasticsearch {
hosts => ["https://localhost:9200/"]
user => "elastic"
password => "test"
dlq_custom_codes => [401]
data_stream => "true"
}
}
Provide logs (if relevant):
[2023-05-22T16:10:21,618][ERROR][logstash.outputs.elasticsearch][main] Invalid data stream configuration, the following parameters are not supported: {"dlq_custom_codes"=>[401]}
[2023-05-22T16:10:21,619][ERROR][logstash.javapipeline ][main] Pipeline error {:pipeline_id=>"main", :exception=>#<LogStash::ConfigurationError: Invalid data stream configuration: ["dlq_custom_codes"]>, :backtrace=>["/Users/user/Documents/elastic/logstash-8.7.1/vendor/bundle/jruby/2.6.0/gems/logstash-output-elasticsearch-11.13.1-java/lib/logstash/outputs/elasticsearch/data_stream_support.rb:76:in `check_enabled_data_stream_config!'", "/Users/user/Documents/elastic/logstash-8.7.1/vendor/bundle/jruby/2.6.0/gems/logstash-output-elasticsearch-11.13.1-java/lib/logstash/outputs/elasticsearch/data_stream_support.rb:59:in `check_data_stream_config!'", "/Users/user/Documents/elastic/logstash-8.7.1/vendor/bundle/jruby/2.6.0/gems/logstash-output-elasticsearch-11.13.1-java/lib/logstash/outputs/elasticsearch/data_stream_support.rb:32:in `data_stream_config?'", "/Users/user/Documents/elastic/logstash-8.7.1/vendor/bundle/jruby/2.6.0/gems/logstash-output-elasticsearch-11.13.1-java/lib/logstash/outputs/elasticsearch.rb:307:in `register'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:69:in `register'", "/Users/user/Documents/elastic/logstash-8.7.1/logstash-core/lib/logstash/java_pipeline.rb:237:in `block in register_plugins'", "org/jruby/RubyArray.java:1865:in `each'", "/Users/user/Documents/elastic/logstash-8.7.1/logstash-core/lib/logstash/java_pipeline.rb:236:in `register_plugins'", "/Users/user/Documents/elastic/logstash-8.7.1/logstash-core/lib/logstash/java_pipeline.rb:607:in `maybe_setup_out_plugins'", "/Users/user/Documents/elastic/logstash-8.7.1/logstash-core/lib/logstash/java_pipeline.rb:249:in `start_workers'", "/Users/user/Documents/elastic/logstash-8.7.1/logstash-core/lib/logstash/java_pipeline.rb:194:in `run'", "/Users/user/Documents/elastic/logstash-8.7.1/logstash-core/lib/logstash/java_pipeline.rb:146:in `block in start'"], "pipeline.sources"=>["/Users/user/Documents/elastic/logstash-8.7.1/config/logstash-dlq.conf"], :thread=>"#<Thread:0x245522bb@/Users/user/Documents/elastic/logstash-8.7.1/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}