OpenSSH sshd disconnects with invalid packet type: 192 on sshpiper >= 1.2.4
Closed this issue · 5 comments
Hi,
I'm facing an issue with sshpiper >= 1.2.4 where sshd receives a packet with type 192 and immediately disconnects afterwards with "connection from 1.2.3.4:11282 closed reason: ssh: disconnect, reason2: Invalid ssh2 packet type: 192".
sshd-backend logs for disconnect:
debug3: receive packet: type 192
debug2: sshpkt_disconnect: sending SSH2_MSG_DISCONNECT: Invalid ssh2 packet type: 192
debug3: send packet: type 1
ssh_dispatch_run_fatal: Connection from user test 10.42.0.59 port 32950: Protocol error
The disconnects happens right after I type something after connecting. Means: Connection works, I get a terminal from the backend but as soon as I type something (press enter, typing a letter) I get disconnected.
SFTP on the other hand works - i can upload/download/delete files, create directories, etc.
Using connection multiplexing with the ControlSocket option and first connecting with SFTP and then multiplexing this connection with SSH I can also use the terminal without disconnects.
So something must be happening when using SSH and trying to send a keystroke for the first time.
With sshpiper 1.2.3 I can't reproduce the error anymore. Digging through the changelog from 1.2.3 to 1.2.4 I found this change in crypto module: tg123/sshpiper.crypto@833695f
Setup
Currently sshpiper is running on a single-node rke2 Kubernetes with this helm chart.
I've installed it with these settings: helm install --create-namespace --namespace sshpiper sshpiper sshpiper/sshpiper --version 0.3.4 --set service.type=LoadBalancer --set service.port=22 --set sshpiper.loglevel=trace --set sshpiper.kubernetes_all_namespaces=true --set rbac.clusterRole=true --set sshpiper.drop_hostkeys_message=true
As backend I'm using a debian:12 base with openssh-9.2p1. I've also tried with lscr.io/linuxserver/openssh-server:latest and got the same disconnects.
I'm not doing anything "fancy" outside of the examples in doc to get the Pipes in k8s working, but I can still post more details about the setup if needed for reproduction.
this is an interesting issue
during handshake, sshpiper replied with ping support to downstream while upstream (sshd) does not support it
opensshd added ping support at version 9.5.
workaround is to upgrade your sshd or use 1.2.3- sshpiper
let me think about how to deal it
btw what is fancy os dist? your ssh client is too new
btw what is fancy os dist? your ssh client is too
new
I'm using Manjaro which already distributes OpenSSH 9.6.
this is an interesting issue during handshake, sshpiper replied with ping support to downstream while upstream (sshd) does not support it opensshd added ping support at version 9.5.
workaround is to upgrade your sshd or use 1.2.3- sshpiper
let me think about how to deal it
If I undestand this correctly the ping@openssh message may either be replied to with the exact copy of the data or may just be ignored.
So one way would be for sshpiper to filter ping@openssh messages if the upstream doesn't support it or reply by itself instead of passing it to upstream.
Though the latter wouldn't reflect the real RTT from backend to upstream which would make the metric kind of irrelevant.
i am not into filtering it as inspecting every single msg contributes to latency
the design of sshpiper is to void touching msg after authentication
however, there was already a filter for msg (drophostkey, off by default)
i am thinking about adding option for ping, but i do not wish to look into every piped msg
chart 0.3.5 published, it can unblock you for now
thanks for reporting this fancy issue