The result of Float type is not unique
Closed this issue · 2 comments
medcl commented
as reported from here:https://discuss.elastic.co/t/logstash-fingerprint/42327
The filter doesn't work if the data type is float,and the recreation is below:
1.create a test file /tmp/test_file
with content
{"b":{"b1":1.1}}
2.use this logstash config
input {
file {
path => ["/tmp/test_file"]
sincedb_path => "/tmp/sincedb"
codec => json
type => "nginx_visit"
start_position => "beginning"
}
}
filter {
fingerprint {
method => "SHA1"
key => "160129"
source => ["[b]"]
}
}
output {
stdout { codec => json }
}
3.the result output
{"b":{"b1":1.1},"@version":"1","@timestamp":"2016-03-08T04:03:24.575Z","path":"/tmp/test_file","host":"medcls-MacBook.local","type":"nginx_visit","fingerprint":"00e4536734914ce8beb46480d14601ed473fb849"}
{"b":{"b1":1.1},"@version":"1","@timestamp":"2016-03-08T04:03:33.600Z","path":"/tmp/test_file","host":"medcls-MacBook.local","type":"nginx_visit","fingerprint":"04373a842b7b13daafd4519c3e5e430082acdc43"}
lucabelluccini commented
Tried both pipelines below, cannot reproduce.
Pipeline:
input {
generator {
lines => [
'{"b":{"b1":1.1}}',
'{"b":{"b1":1.1}}'
]
count => 1
codec => json
}
}
filter {
fingerprint {
method => "SHA1"
key => "160129"
source => ["[b]"]
}
}
output {
stdout {
codec => json
}
}
Output:
{"fingerprint":"5598acd401c93680b57020d49dfd89323b3b0a3a","host":"PC","@version":"1","@timestamp":"2018-10-12T22:19:43.505Z","b":{"b1":1.1},"sequence":0}{"fingerprint":"5598acd401c93680b57020d49dfd89323b3b0a3a","host":"PC","@version":"1","@timestamp":"2018-10-12T22:19:43.487Z","b":{"b1":1.1},"sequence":0}
Also tested:
{"b":{"b1":1.1}}
{"b":{"b1":1.1}}
Pipeline:
input {
file {
path => "./logstash-6.4.2/test"
sincedb_path => './logstash-6.4.2/sincedb'
codec => json
start_position => "beginning"
}
}
filter {
fingerprint {
method => SHA1
key => '160129'
source => [ '[b]' ]
}
}
output {
stdout {
codec => json
}
# stdout {
# codec => 'rubydebug'
# }
}
Output:
{"@timestamp"=>2018-10-13T14:20:34.990Z, "fingerprint"=>"5598acd401c93680b57020d49dfd89323b3b0a3a", "@version"=>"1", "b"=>{"b1"=>1.1}, "host"=>"LUCA-PC", "path"=>"----"}
{"@timestamp":"2018-10-13T14:20:34.990Z","fingerprint":"5598acd401c93680b57020d49dfd89323b3b0a3a","@version":"1","b":{"b1":1.1},"host":"LUCA-PC","path":"----"}
medcl commented
maybe fixed, closing now.