logstash-plugins/logstash-input-couchdb_changes

couchdb_changes doesn't close when shutting down logstash?

Opened this issue · 4 comments

FlorinAndrei commented

See elastic/logstash#4703

logstash-2.2.2, elasticsearch-1.7.5, java-1.8.0_72-b15, ubuntu-14.04 64bit, couchdb-1.6.1

input {
  couchdb_changes {
    always_reconnect => true
    db => "vendnext_new_no_auth"
    heartbeat => 5000
    host => "couch31-prod-uswest2-aws.domain"
    port => 5984
    reconnect_delay => 10
  }
}
filter {
  mutate {
    remove_field => [ "stripe", "moneris", "roles", "_rev", "freeVendUses", "surcharge", "debugMessages" ]
    convert => [ "[doc][debugMessages]", "string"]
  }
}
output {
  elasticsearch {
    action => "index"
    hosts => "localhost:9200"
    index => "vendnext_new_no_auth_index"
    manage_template => false
    script => "domain-filter"
    script_lang => "javascript"
    script_type => "file"
  }
}

Overall it appears to work just fine. But the problems begin when I try to stop it.

"service logstash stop" stops the updates, but the logstash process does not quit. Repeating "service logstash stop" makes no difference. Logstash appears to be frozen - no updates sent to ElasticSearch, but it does not quit either.

After that, "service logstash status" reports the service as still running.

"kill $PID" doesn't kill it. The only way to kill it is "kill -9 $PID".

teebu commented

Same here. logstash 2.3.4, logstash-input-couchdb_changes (2.0.4)

  • 👍1
DrRob commented

Me too when running logstash 5.0.1 directly (not as a service), and the log says:

[2016-11-20T04:38:50,809][WARN ][logstash.runner          ] SIGINT received. Shutting down the agent.
[2016-11-20T04:38:50,815][WARN ][logstash.agent           ] stopping pipeline {:id=>"main"}
[2016-11-20T04:38:55,816][WARN ][logstash.runner          ] Received shutdown signal, but pipeline is still waiting for in-flight events
to be processed. Sending another ^C will force quit Logstash, but this may cause
data loss.
[2016-11-20T04:40:56,893][WARN ][logstash.shutdownwatcher ] {}
[2016-11-20T04:40:56,894][ERROR][logstash.shutdownwatcher ] The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information
  • 👍1
uschtwill commented

Same for me running it on Docker and using docker stop -t 180 logstash. :/

kingigi commented

I am running LogStash 6.0.1 on Fedora 26 and I am seeing the same problem.
I was hoping that shutting down CouchDB would help (to force a TCP-disconnect), but it doesn't.
When I stop LogStash (/bin/systemctl stop logstash.service) and stop CouchDB after that, the plugin even keeps trying to reconnect to CouchDB.