port 5514 not listening
Closed this issue · 7 comments
Hello @ChristianKniep -
Ran into an issue I'm having trouble unwinding:
I'm unable to send anything over 5514 to a new docker host I have built out- and when I spawn the elk container, it does not look like anything is starting up a port to listen on 5514:
[root@fa503a62eeac nginx]# supervisor_daemonize.sh
Error: Another program is already listening on a port that one of our HTTP servers is configured to use. Shut this program down first before starting supervisord.
For help, use /usr/bin/supervisord -h
# supervisorctl status
consul RUNNING pid 13, uptime 0:02:03
diamond RUNNING pid 12, uptime 0:02:03
elasticsearch RUNNING pid 8, uptime 0:02:03
logstash RUNNING pid 1838, uptime 0:00:03
logstash_watchdog RUNNING pid 9, uptime 0:02:03
nginx RUNNING pid 16, uptime 0:02:03
sshd RUNNING pid 11, uptime 0:02:03
statsd RUNNING pid 10, uptime 0:02:03
syslog-ng RUNNING pid 14, uptime 0:02:03
[root@fa503a62eeac nginx]# lsof -i -n | egrep 'COMMAND|LISTEN'
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
java 8 root 234u IPv6 393279 0t0 TCP *:vrace (LISTEN)
java 8 root 363u IPv6 395313 0t0 TCP *:wap-wsp (LISTEN)
sshd 11 root 3u IPv4 367506 0t0 TCP *:ssh (LISTEN)
sshd 11 root 4u IPv6 367508 0t0 TCP *:ssh (LISTEN)
statsd 20 root 10u IPv4 393259 0t0 TCP *:8126 (LISTEN)
nginx 25 root 8u IPv4 388535 0t0 TCP 127.0.0.1:webcache (LISTEN)
nginx 25 root 9u IPv4 388536 0t0 TCP *:http (LISTEN)
syslog-ng 42 root 9u IPv4 388531 0t0 TCP *:shell (LISTEN)
consul 132 root 3u IPv6 367517 0t0 TCP *:tmi (LISTEN)
consul 132 root 12u IPv6 367521 0t0 TCP *:amberon (LISTEN)
consul 132 root 15u IPv6 367523 0t0 TCP *:8302 (LISTEN)
consul 132 root 17u IPv6 367525 0t0 TCP *:cvd (LISTEN)
consul 132 root 18u IPv6 367526 0t0 TCP *:fmtp (LISTEN)
consul 132 root 24u IPv6 388540 0t0 TCP *:domain (LISTEN)
nginx 140 nginx 8u IPv4 388535 0t0 TCP 127.0.0.1:webcache (LISTEN)
nginx 140 nginx 9u IPv4 388536 0t0 TCP *:http (LISTEN)
On another host that is working:
[root@243c7789ba73 opt]# lsof -i -n | egrep 'COMMAND|LISTEN'
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
java 9 root 227u IPv6 24062 0t0 TCP *:vrace (LISTEN)
java 9 root 358u IPv6 24106 0t0 TCP *:wap-wsp (LISTEN)
python 13 root 6u IPv4 14848 0t0 TCP *:hsl-storm (LISTEN)
python 13 root 7u IPv4 14849 0t0 TCP *:newheights (LISTEN)
python 13 root 8u IPv4 14850 0t0 TCP *:talon-engine (LISTEN)
python 14 root 6u IPv4 12679 0t0 TCP *:zephyr-clt (LISTEN)
python 14 root 7u IPv4 12680 0t0 TCP *:zephyr-hm (LISTEN)
python 14 root 8u IPv4 12681 0t0 TCP *:afs3-prserver (LISTEN)
gunicorn 18 root 5u IPv4 23926 0t0 TCP 127.0.0.1:distinct (LISTEN)
nginx 22 root 8u IPv4 15646 0t0 TCP 127.0.0.1:websm (LISTEN)
nginx 22 root 9u IPv4 15647 0t0 TCP *:webcache (LISTEN)
nginx 22 root 10u IPv4 15648 0t0 TCP *:http (LISTEN)
python 23 root 6u IPv4 725 0t0 TCP *:cfinger (LISTEN)
python 23 root 7u IPv4 726 0t0 TCP *:mailbox (LISTEN)
nginx 24 root 6u IPv4 17845 0t0 TCP *:ddi-tcp-1 (LISTEN)
java 25 root 26u IPv6 20049 0t0 TCP *:shell (LISTEN)
java 25 root 37u IPv6 24113 0t0 TCP *:5514 (LISTEN)
java 25 root 242u IPv6 25757 0t0 TCP *:9301 (LISTEN)
nginx 55 nginx 6u IPv4 17845 0t0 TCP *:ddi-tcp-1 (LISTEN)
nginx 56 nginx 8u IPv4 15646 0t0 TCP 127.0.0.1:websm (LISTEN)
nginx 56 nginx 9u IPv4 15647 0t0 TCP *:webcache (LISTEN)
nginx 56 nginx 10u IPv4 15648 0t0 TCP *:http (LISTEN)
gunicorn 99 root 5u IPv4 23926 0t0 TCP 127.0.0.1:distinct (LISTEN)
gunicorn 107 root 5u IPv4 23926 0t0 TCP 127.0.0.1:distinct (LISTEN)
sshd 124 root 3u IPv4 18705 0t0 TCP *:ssh (LISTEN)
sshd 124 root 4u IPv6 18707 0t0 TCP *:ssh (LISTEN)
Java is opening a port to listen on 5514.
Have you seen this behavior before?
Hey @kotter,
you are not able to send it to a new logstash host? The logstash process seems to be crashing.
It's up only for 3sec while the other supervisor processes are up for over two minutes.
What does the logfile say (/var/log/supervisor/logstash.log)?
Thanks @ChristianKniep - looks like I gained a few line breaks when transferring my logstash.conf file out of the old instance.
/check your configs!
it seems it's not intuitive enough... Should think of a way to make this error more obvious...
yeah, I had to do a little searching around and add lsof to the image temporarily to try to find out why I was getting a rejection... if I had just hit the /var/log/supervisor/logstash.log (maybe add that to your doc if it's not already there?) I would have seen the error almost immediately. More my ignorance of the log location than any issue with what you have produced.
Thanks again!
one way would be to add a step to the startscript that does a config-check (logstash has an option for that). If that fails you got a differnent error message. I'll keep that in mind and sleep over it. :)
ahh.. wrong button...
one way would be to add a step to the startscript that does a config-check (logstash has an option for that). If that fails you got a differnent error message. I'll keep that in mind and sleep over it. :)
👍