Add option to set time interval between paging queries

[JooHyukKim]

Table Of Contents

1. My situation
2. What I want
3. How I can help

1. my situation

1. I have to run logstash jdbc input against Database running in production.
2. I use pagination options to limit amount of data per query
3. but since SQL statements are run almost non-stop
4. Database metrics spike, affecting performance of other functionality
5. Slow down the logstash-input-jdbc queries

2. What I want : new option called something like jdbc_paging_interval_in_millis

• allow user to control time sleep in betweeen running SQL statements with pagination
• only applied when jdbc_paging_enabled => true is set
• user will consider this new option as performance tradeoff between performance of LogStash pipeline and performance of target database

Workflow diagram

Expected Outcome : new configuration option

3. How I can help

if this sounds workable & reasonable feature, I will send a PR

[JooHyukKim]

Attention

• meanwhile, I solved my problem by discussing this issue on Elastic Community.
• there is filter plugin called sleep to sleep pipeline workers
• when using sleep filter plugin, be mindful of LogStash's execution model(Thread Model) which you can manipulate using LogStash's pipeline.workers and pipeline.batch.size settings mentioned in Official Elastic
• or send me an email to help you out

cheers...!