logging back after a broken pipe
Closed this issue · 7 comments
Hi,
I got the following error when logging back to my ssh server after I got a broken pipe ssh connection (probably due to time out). The username is my username on the cluster I logged into.
Warning: remote port forwarding failed for listen port 52698
Then when I use rmate to open a file, it gives:
/home/username/bin/rmate:174:in `readline': end of file reached (EOFError)
from /home/username/bin/rmate:174:in `connect_and_handle_cmds'
from /home/username/bin/rmate:223
from /home/username/bin/rmate:222:in `fork'
from /home/username/bin/rmate:222
netstat -ltnp | grep 52698 gives the following:
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp 0 0 127.0.0.1:52698 0.0.0.0:* LISTEN -
tcp 0 0 ::1:52698 :::* LISTEN -
It seems that I cannot see the pid of those connections. I am just wondering if there's any way to close those dead connections so that I can use rmate again.
I know that logging out completely and waiting for a while will get those connections wiped out on the cluster. But I just wonder if there's a more efficient way to addressing the problem.
I have been using remote editing with textmate by ssh tunnel method for a while. (Thanks to it, I was able to resume my work anywhere with internet access.)
I would almost think that this is related to ssh since that is responsible for setting up the listening port on localhost.
But if the port is not released because of rmate then try grep your process list for ruby as that is likely what rmate will appear under (i.e. the interpreter used to run the script).
Hi @sorbits
Thanks for the reply. I tried grepping ruby but didn't get any ruby process. perhaps because the port is
opened by ssh process? Or I need sudo to see the process name/id of it?
Below is the searching result after I came across this problem again.
$ netstat -ltnp
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:33944 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:51417 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:52698 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:6010 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:6011 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:6012 0.0.0.0:* LISTEN -
tcp 0 0 :::111 :::* LISTEN -
tcp 0 0 :::22 :::* LISTEN -
tcp 0 0 ::1:25 :::* LISTEN -
tcp 0 0 ::1:52698 :::* LISTEN -
tcp 0 0 ::1:6010 :::* LISTEN -
tcp 0 0 ::1:6011 :::* LISTEN -
tcp 0 0 ::1:6012 :::* LISTEN -
Should I try ps aux | grep ruby instead?
As you are using an ssh tunnel it is not rmate which keeps the connection between your Mac and server, as confirmed by the lack of ruby processes, but rather ssh.
Why ssh blocks the port even after the connection is lost, I do not know, but you can try autossh which seems to be created for your situation (with an unstable connection).
Just figured out a way to close the port used by previous ssh session:
$ ssh cluster
Warning: remote port forwarding failed for listen port 52698
username@cluster ~ $ ps -u username
PID TTY TIME CMD
10901 ? 00:00:00 sshd
10906 pts/1 00:00:00 bash
13066 ? 00:00:00 sshd
13071 pts/14 00:00:00 bash
13124 pts/14 00:00:00 ps
username@cluster ~ $ kill 10901
username@cluster ~ $ kill 10906
-bash: kill: (10906) - No such process
Then I log back again. The warning disappears and rmate functions again.
👍
I was so tired of this happening all the time since moving to Brazil (horrible connectivity) that I made simple script to kill all existing ssh connections. Of course this will also kill my active connection! It could be improved by excluding current sessions' sshd pid.
#!/bin/sh
sudo killall -u YOURUSER -r sshd
If only there were a way to run this over mosh... that would be amazing.