Paging bug: JDBC_PAGING_ENABLED and JDBC_PAGE_SIZE degraded after version 4.3.14
Closed this issue · 0 comments
bmax commented
Hello! It seems like at the same time prepared statements were introduced in version 4.3.16 is the same time paging functionality broke. I've tested this by rolling back to 4.3.14 and ensuring that jdbc_paging_enabled
works. I am also working on a PR for a fix but in the mean time I thought I'd get this issue out first in case anyone else is having trouble.
The problem resides here: https://github.com/logstash-plugins/logstash-input-jdbc/blob/master/lib/logstash/plugin_mixins/jdbc/statement_handler.rb#L34
The variable @jdbc_paging_enabled
is not available in that context therefore this will never evaluate as true.
Debugging information
- Version: > 4.3.14
- Operating System: centos (Docker image logstash:7.4.1)
- Config File (if you have sensitive info, please remove it):
input {
jdbc {
jdbc_driver_library => "/usr/share/logstash/bin/postgresql-42.2.8.jar"
jdbc_driver_class => "org.postgresql.Driver"
jdbc_connection_string => "jdbc:postgresql://${POSTGRES_URL}/${POSTGRES_DB}"
jdbc_user => "${POSTGRES_USER}"
jdbc_password => "${POSTGRES_PASSWORD}"
schedule => "* * * * * *"
jdbc_paging_enabled => true
jdbc_page_size => 10
statement => "SELECT * from test"
}
}
output {
elasticsearch {
document_id => "visit-%{user_id}-%{merchant_id}"
hosts => [ "${ELASTICSEARCH_HOST}" ]
index => "program_users"
}
}
- Steps to Reproduce: Include
jdbc_paging_enabled
and ensure that the select query does not include limit or offset.