logstash-plugins/logstash-filter-geoip

Add `interval` option to config auto-update schedule

kaisecheng opened this issue · 2 comments

Since 7.14, geoip check database updates every 24h since the server start time. The suggestion is to add interval option to allow users to change the checking interval in hours. The number should be >= 24 and < 168 (7 days)

xpack.geoip.download.interval=72

open question

  1. Is the interval only for database update check? Is the expiry routine running every day to respect the 30 days expiry rule?
  2. What is the motivation of have interval config? Checking updates every three days can save some server load considering users deploy ten thousand instances.
  1. elasticsearch check if db is valid on every use of geoip processor and tag doc if that’s not the case. cleanup of expired databases is done with ingest.geoip.downloader.poll.interval
  2. Check every 3 days is to reduce bandwidth, it’s configurable to allow for checking more aggressive (every day) if customer wants to always have freshest data possible or for tests where it’s easier to change this interval and wait seconds instead of days