logstash-plugins/logstash-input-couchdb_changes

Update speed degrades over time

Closed this issue · 1 comments

konradkonrad commented

Hi there!
I am using v0.1.4 with logstash 1.5.0rc2 for ~10 days now.

My configuration:

  • I have a two couchdb_changes inputs for two databases (small and big) on the same (remote) couchdb.
  • Output into elasticsearch (same mapping).

In the first 1-2 days, the small db (overall ~15mio seq events) got up-to-date, big (~250mio seq events) synced around ~45mio seq events.

edit: so speed was at least ~30,000 seq events per minute during that period.

I did not monitor the progress closely after the first 2 days.
When I checked back today, small was still up-to-date, but the update speed for big was down to only ~1000 events per minute, overall it was only at ~60mio.

The logstash agent is started with --debug, but there was nothing in the log/output. The elasticsearch target (5 data-nodes) seemed stable (load < 0.5, HEAP "okay").

After restarting logstash, the updating speed went up to ~200,000 seq events per minute.

untergeek commented

Closing this as it's so old, with no updates.