logstash-plugins/logstash-input-file

SinceDB doesn't work when file input path has spaces

Closed this issue · 1 comments

This is against v4.1.10 on any operating system.

Steps to reproduce:

  • Create a simple file input pipeline where path has no spaces, for example:
input {
  file {
    path => '/home/ubuntu/logs/logstash_plain.log'
    start_position => "beginning"
    sincedb_path => "sinceDbLogstashLogs"
  }
}
output {
  file {
    path => "/tmp/lsTest"
  }
}

Then, run logstash on this conf file, note the line count of /tmp/lsTest, then run logstash on this conf file again, and then note that the line count of /tmp/lsTest does not change (nor is /tmp/lsTest even opened).

Now, make a log file with a space in its name, update the conf file to point path at this file (e.x., path => '/home/ubuntu/logs/logstash plain.log'), run logstash once, note the line count of /tmp/lsTest, then run it again, and now note that the line count of /tmp/lsTest has doubled.

This very clearly seems to be a culprit of this line:
https://github.com/logstash-plugins/logstash-input-file/blob/master/lib/filewatch/sincedb_record_serializer.rb#L41

This is made more likely by the fact that when you delete the path and access time fields in sinceDB so that it uses the old format (<inode> <majorFs> <minorFs> <byteOffset>), this problem goes away.

Fix in #249 was merged and is now available in v4.1.13.