Failing to provide a + in the date pattern of the index property silently fails.
nicenemo opened this issue · 0 comments
nicenemo commented
Logstash information:
- Using bundled JDK: /usr/share/logstash/jdk logstash 8.4.1 But also fails on older versions
- Debian packages from Elastic. But also fails on Mac.
- Either as service or as command line
- plugin came packaged with logstash
We run Logstash both as systemd service as well as commandline tool.
We run on Debian Bullseye and MacOS.
Debian is ran on AWS ARM & Intel based systems. It is run on Windows WSL2 based Debian instances too.
JVM (e.g. java -version
):
Bundled
Steps to reproduce:
Use a config like this:
input {
# We read from backup files. Note that the start of the path MUST be absolute
file {
path => "/opt/data/backup/ecorp-fluffy-data-2022.33.json"
sincedb_path => "/tmp/sincedb_restore-ecorp.db"
start_position => "beginning"
file_chunk_size => 268435456
mode => "read"
codec => "json"
file_completed_action => "log"
file_completed_log_path => "/opt/data/logstash/logs/restore-ecorp-fluffy-data.log"
}
}
filter {
if "_jsonparsefailure" in [tags] {
drop { }
}
# sort
json {
source => "message"
}
}
output {
# We write to the "new" cluster
elasticsearch {
manage_template => false
sniffing => false
ilm_enabled => false
# does not work and fails silent because of missing +.
index => "ecorp-fluffy-%{YYYY.MM}.01"
# Does work. the added fixed 01 is to work around another bug/feature
# index => "ecorp-fluffy-%{+YYYY.MM}.01"
doc_as_upsert => true
action => "update"
document_id => "%{identifier}"
http_compression => true
ssl => true
ssl_certificate_verification => false
hosts => ["https://localhost:9200"]
api_key => " not going to tell you"
}
# We print dots to see it in action
stdout {
codec => "dots"
}
}
It will fail silently.
Provide logs (if relevant):
That's the problem.... Logstash is silent ;)